dolphinscheduler的僵尸任务清理

朝颜。

已于 2022-09-05 15:41:34 修改

阅读量1.5k

点赞数

文章标签： sql dolphin hadoop

于 2022-06-20 17:36:58 首次发布

原文链接：https://blog.csdn.net/u010978399/article/details/123477244

版权

本文介绍了一位工程师如何通过SQL查询和删除操作，从海豚调度的数据库中清理数百条僵尸任务。首先，通过end_time为空的条件删除大部分任务，然后针对特定日期的工作流实例、定义、任务和历史任务进行删除。接着，清除了数据源、工作空间管理相关的数据，并涉及数据清理的多个方面，如用户、租户、警告邮件地址、令牌等。整个过程详细记录了每个删除操作的SQL查询，旨在彻底解决僵尸任务问题。

摘要由CSDN通过智能技术生成

原因：海豚调度中有几百条僵尸任务，界面怎么也删不掉，想从数据库中删除，开始查找从数据库删除的办法。

参考以下脚本，结合我库中僵尸任务的特征，用end_time is null删除了大半。
– 删除任务实力中运行的任务
select * from t_ds_process_instance where end_time is null and command_type=9

实在找不到特征的，就按名字删除了。

– 删除任务实力中运行的任务
select * from t_ds_process_instance where end_time is null and command_type=9 – state=6 end_time is null

#删除工作流实例
select * from t_ds_process_instance where DATE_FORMAT(start_time ,“%Y-%m-%d”) = ‘2021-06-23’;

delete from t_ds_process_instance where DATE_FORMAT(start_time ,“%Y-%m-%d”) = ‘2021-06-23’ and state = ‘1’;

#删除工作流
select * from t_ds_process_definition where DATE_FORMAT(create_time ,“%Y-%m-%d”) = ‘2021-06-23’;

delete from t_ds_process_definition where DATE_FORMAT(create_time,“%Y-%m-%d”) = ‘2021-06-23’;

#删除任务和历史任务
select * from dk_hc_task ;

delete from dk_hc_task ;

#历史任务
select * from dk_hc_history_log ;

delete from dk_hc_history_log ;

#删除节点
select * from dk_hc_workflow ;

delete from dk_hc_workflow ;

#删除节点承载的目录
select * from dk_hc_workflow_folder ;

delete from dk_hc_workflow_folder where title !=‘根目录’;

INSERT INTO dk_hc_workflow_folder
(id, title, wf_id, create_time, create_user, update_time, update_user, parent_id, type, expand, contextmenu, project_id, edit_user, flowType)
VALUES(1, ‘根目录’, NULL, ‘2021-02-02 15:51:29.0’, 1, NULL, 1, 0, 1, 1, 1, 0, 1, NULL);

#删除临时表
select * from t_ds_command ;

delete from t_ds_command ;

delete from t_ds_error_command ;

#清除数据源
select * from t_ds_datasource ;

delete from t_ds_datasource where name != ‘192.168.111.222’ ;

#---------------------------------------工作空间管理----------------------------------------------------------

#删除租户
select * from t_ds_tenant where tenant_code != ‘root’;

delete from t_ds_tenant where tenant_code != ‘root’;

#删除用户
select * from t_ds_user where user_name != ‘admin’;

delete from t_ds_user where user_name != ‘admin’;

#删除worker分组，default不在数据之内
select * from t_ds_worker_group ;

delete from t_ds_worker_group;

#删除警告邮件地址
select * from t_ds_alertgroup ;

delete from t_ds_alertgroup;

#删除令牌
select * from t_ds_access_token ;

delete from t_ds_access_token;

#删除队列
select * from t_ds_queue where queue_name !=‘default’;

delete from t_ds_queue where queue_name !=‘default’;

#删除项目空间

select * from t_ds_project ;

delete from t_ds_project where name !=‘临时调度’;

#---------------------------------------工作空间管理----------------------------------------------------------

#清除datax的记录
select * from dk_datax_statistics ;

delete from dk_datax_statistics;
————————————————
版权声明：本文为CSDN博主「吃素的哈士奇」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/u010978399/article/details/123477244