升级前备份和准备:
备份海豚元数据库
提前准备好相对应的jar包。如Mysql驱动等(8.0.16版本)
海豚升级步骤:
官网升级步骤地址:https://dolphinscheduler.apache.org/zh-cn/docs/latest/user_doc/guide/upgrade.html
升级还是比较容易的只需要改一个文件、执行两个脚本即可。
官方2.0.5下载地址:https://dlcdn.apache.org/dolphinscheduler/2.0.5/apache-dolphinscheduler-2.0.5-bin.tar.gz
1、下载到服务器
wget https://dlcdn.apache.org/dolphinscheduler/2.0.5/apache-dolphinscheduler-2.0.5-bin.tar.gz
2、解压缩
tar -zxvf apache-dolphinscheduler-2.0.5-bin.tar.gz
3、更新当前海豚环境,按照老版本配置修改以下两个文件
vim apache-dolphinscheduler-2.0.5-bin/conf/config/install_config.conf
vim apache-dolphinscheduler-2.0.5-bin/conf/env/dolphinscheduler_env.sh
4、拷贝mysql驱动包,3.1.1版本目录结构发生变更,如下图
需要将mysql驱动包拷贝至master节点:alert-server,api-server,master-server
worker节点:worker-server目录下的libs目录中
5、需要将各libs目录下的curator*.jar 替换成dolphinschedule2.0.5版本对应的jar包,另外生产环境的zookeeper版本如果是3.5.0以下的也需要将zookeeper的相关jar包替换dolphinschedule2.0.5版本的zookeeper包
以上差不多就是dolphinschedule2.0.5升级dolphinschedule3.1.1中的过程
可能出现的问题:
[ERROR] 2022-06-13 17:59:45.280 org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteThread:[244] - handler error:
org.mybatis.spring.MyBatisSystemException: nested exception is org.apache.ibatis.exceptions.TooManyResultsException: Expected one result (or null) to be returned by selectOne(), but found: 2
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:78)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:440)
at com.sun.proxy.$Proxy91.selectOne(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectOne(SqlSessionTemplate.java:159)
at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:89)
at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:61)
at com.sun.proxy.$Proxy115.queryByDefinitionCodeAndVersion(Unknown Source)
at sun.reflect.GeneratedMethodAccessor124.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.cache.interceptor.CacheInterceptor.lambda$invoke$0(CacheInterceptor.java:54)
at org.springframework.cache.interceptor.CacheAspectSupport.invokeOperation(CacheAspectSupport.java:366)
at org.springframework.cache.interceptor.CacheAspectSupport.lambda$handleSynchronizedGet$1(CacheAspectSupport.java:447)
at org.springframework.cache.support.NoOpCache.get(NoOpCache.java:76)
at org.springframework.cache.interceptor.CacheAspectSupport.handleSynchronizedGet(CacheAspectSupport.java:442)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:382)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:345)
at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:64)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy116.queryByDefinitionCodeAndVersion(Unknown Source)
at org.apache.dolphinscheduler.service.process.ProcessService.findTaskDefinition(ProcessService.java:2474)
at org.apache.dolphinscheduler.service.process.ProcessService.lambda$getTaskDefineLogListByRelation$4(ProcessService.java:2465)
at java.util.HashMap.forEach(HashMap.java:1291)
at org.apache.dolphinscheduler.service.process.ProcessService.getTaskDefineLogListByRelation(ProcessService.java:2464)
at org.apache.dolphinscheduler.service.process.ProcessService$$FastClassBySpringCGLIB$$ed138739.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.apache.dolphinscheduler.service.process.ProcessService$$EnhancerBySpringCGLIB$$ec695045.getTaskDefineLogListByRelation(<generated>)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteThread.buildFlowDag(WorkflowExecuteThread.java:578)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteThread.startProcess(WorkflowExecuteThread.java:542)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteThread.run(WorkflowExecuteThread.java:239)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ibatis.exceptions.TooManyResultsException: Expected one result (or null) to be returned by selectOne(), but found: 2
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectOne(DefaultSqlSession.java:80)
at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:426)
... 42 common frames omitted
关键信息 Expected one result (or null) to be returned by selectOne(), but found: 2,即sql的查询逻辑是结果为1条,但是却出现2条的情况
定位源码中报错的位置
at org.apache.dolphinscheduler.service.process.ProcessService.findTaskDefinition(ProcessService.java:2474)
解决办法:
在dolphinsscheduler对应的元数据库中,查询t_ds_task_definition_log表
SELECT code ,version,COUNT(*) cnt from t_ds_task_definition_log group by code ,version order by cnt desc
将重复的数据 条数删除即可
详情可参考:【异常解决】DolphinScheduler-2.0.5 工作流实例无法调度和停止异常_北溟小鱼123的博客-CSDN博客