DolphinScheduler Ambari插件解说

1，stacks定义标准
statcks中定义了HDP的不同的版本，如果想自定义一个插件，必须首先要明了此插件是放在HDP那个版本中的，比如下图是放在3.1版本中的，
在3.1下建立自己的插件名称，然后自定义建立 metainfo.xml 文件，该文件引用了common-services中的指定的版本

2，common-services定义
common-services 定义了插件的不同的版本，而且不同的版本是可以有继承关系的。

具体common-services 定义如下：

2.1 metainfo.xml是用来描述服务的

2.0



DOLPHIN

Dolphin Scheduler
分布式易扩展的可视化DAG工作流任务调度系统
1.3.3

        <!-- components下每个 component 表示一个服务 -->
        <components>
            <component>
                <name>DOLPHIN_MASTER</name>
                <displayName>DS Master</displayName>


				<!-- category 指定了该模块（Component）的类别，可以是 MASTER、SLAVE、CLIENT -->
                <category>MASTER</category>
				
				<!-- 指的是所要安装的机器数，可以是固定数字 1，可以是一个范围比如 1-2，也可以是 1+，或者 ALL。如果是一个范围的时候，安装的时候会让用户选择机器 -->
                <cardinality>1+</cardinality>
                <commandScript>


					<!-- 服务对应的脚本，控制比如说： install 、start、stop etc -->
                    <script>scripts/dolphin_master_service.py</script>
                    <scriptType>PYTHON</scriptType>
                    <timeout>600</timeout>
                </commandScript>
            </component>

            <component>
                <name>DOLPHIN_LOGGER</name>
                <displayName>DS Logger</displayName>
                <category>SLAVE</category>
                <cardinality>1+</cardinality>
                <commandScript>
                    <script>scripts/dolphin_logger_service.py</script>
                    <scriptType>PYTHON</scriptType>
                    <timeout>600</timeout>
                </commandScript>
            </component>

            <component>
                <name>DOLPHIN_WORKER</name>
                <displayName>DS Worker</displayName>
                <category>SLAVE</category>
                <cardinality>1+</cardinality>
                <dependencies>
                    <dependency>
                        <name>DOLPHIN/DOLPHIN_LOGGER</name>
                        <scope>host</scope>
                        <auto-deploy>
                            <enabled>true</enabled>
                        </auto-deploy>
                    </dependency>
                </dependencies>
                <commandScript>
                    <script>scripts/dolphin_worker_service.py</script>
                    <scriptType>PYTHON</scriptType>
                    <timeout>600</timeout>
                </commandScript>
            </component>

            <component>
                <name>DOLPHIN_ALERT</name>
                <displayName>DS Alert</displayName>
                <category>SLAVE</category>
                <cardinality>1</cardinality>
                <commandScript>
                    <script>scripts/dolphin_alert_service.py</script>
                    <scriptType>PYTHON</scriptType>
                    <timeout>600</timeout>
                </commandScript>
            </component>

            <component>
                <name>DOLPHIN_API</name>
                <displayName>DS_Api</displayName>
                <category>SLAVE</category>
                <cardinality>1</cardinality>
                <commandScript>
                    <script>scripts/dolphin_api_service.py</script>
                    <scriptType>PYTHON</scriptType>
                    <timeout>600</timeout>
                </commandScript>
            </component>
        </components>

		<!--强制依赖的服务-->
        <requiredServices>
            <service>ZOOKEEPER</service>
        </requiredServices>

		<!--
			该标签描述这个组件需要的依赖。Ambari 会使用yum或者apt-get去安装这些依赖。
			osFamily ： 支持 Linux 的发行版本，如 redhat6, suse11, ubuntu12 
			package ：依赖的包，Ambari 会使用yum或者apt-get去安装这些依赖
		--->
        <osSpecifics>
            <osSpecific>
                <osFamily>any</osFamily>
                <packages>
                    <package>
                        <name>apache-dolphinscheduler-incubating*</name>
                    </package>
                </packages>
            </osSpecific>
        </osSpecifics>

		<!-- 对用的configuration xml 配置文件名 --->
        <configuration-dependencies>
            <config-type>dolphin-alert</config-type>
            <config-type>dolphin-app-api</config-type>
            <config-type>dolphin-app-dao</config-type>
            <config-type>dolphin-common</config-type>
            <config-type>dolphin-env</config-type>
            <config-type>dolphin-quartz</config-type>
        </configuration-dependencies>

		<!-- 安装的时候定义的界面 --->
        <themes>
            <theme>
                <fileName>theme.json</fileName>
                <default>true</default>
            </theme>
        </themes>
		
		<!-- URL快速链接跳转页 --->
        <quickLinksConfigurations-dir>quicklinks</quickLinksConfigurations-dir>
        <quickLinksConfigurations>
            <quickLinksConfiguration>
                <fileName>quicklinks.json</fileName>
                <default>true</default>
            </quickLinksConfiguration>
        </quickLinksConfigurations>
    </service>
</services>

1，一个service是一个项目
2，一个service项目下有很多components，每一个component是一个服务
3，service category 指定了该模块（Component）的类别，可以是 MASTER、SLAVE、CLIENT
4，service Cardinality 指的是所要安装的机器数，可以是固定数字 1，可以是一个范围比如 1-2，也可以是 1+，或者 ALL。如果是一个范围的时候，安装的时候会让用户选择机器
5，configuration-dependencies 中设置 xml的文件的依赖
6，themes 设置页面需要传递的参数
7，quickLinksConfigurations-dir，quickLinksConfigurations-dir 配置快速链接地址

2.2 configuration用来存放配置文件
configuration文件夹下主要存放的是一些xml文件，对应着配置文件，定义了配置文件中的变量。
如右图所示，可以让用户进行填写内容，对应的是 ADVANCED

2.3 alerts.json
alerts.json用于服务的定期check，其中有两种类型的服务进行check。
1，一种类型是对于这种网络端口的，比如说http这种活着是rpc这种类型的。
2，还有一种是自己开发的服务，根据pid来判断是否存在
{
“DOLPHIN”: {
“service”: [],
“DOLPHIN_API”: [
{
“name”: “dolphin_api_port_check”,
“label”: “dolphin_api_port_check”,
“description”: “dolphin_api_port_check.”,
“interval”: 10,

“scope”: “ANY”,
“source”: {

“type”: “WEB”,
“uri”: “{{dolphin-application-api/server.port}}”,
“default_port”: 12345,
“reporting”: {
“ok”: {
“text”: “HTTP OK - {0:.3f}s response on port {1}”
},
“warning”: {
“text”: “HTTP OK - {0:.3f}s response on port {1}”,
“value”: 1.5
},
“critical”: {
“text”: “Connection failed: {0} to {1}:{2}”,
“value”: 5.0
}
}
}
}
],
“DOLPHIN_LOGGER”: [
{
“name”: “dolphin_logger_port_check”,
“label”: “dolphin_logger_port_check”,
“description”: “dolphin_logger_port_check.”,
“interval”: 10,
“scope”: “ANY”,
“source”: {
<!-- type 为 PORT , 用来监测机器上的一个端口是否可用 —>
“type”: “PORT”,
“uri”: “{{dolphin-common/loggerserver.rpc.port}}”,
“default_port”: 50051,
“reporting”: {
“ok”: {
“text”: “TCP OK - {0:.3f}s response on port {1}”
},
“warning”: {
“text”: “TCP OK - {0:.3f}s response on port {1}”,
“value”: 1.5
},
“critical”: {
“text”: “Connection failed: {0} to {1}:{2}”,
“value”: 5.0
}
}
}
}
],
“DOLPHIN_MASTER”: [
{
“name”: “DOLPHIN_MASTER_CHECK”,
“label”: “check dolphin scheduler master status”,
“description”: “”,
“interval”:10,

	<!-- 表示只能在本服务机器上 -->
    "scope": "HOST",
    "enabled": true,
    "source": {
      "type": "SCRIPT",
	   <!-- 执行相应的脚步，来check 服务是否正常 -->
      "path": "DOLPHIN/1.3.3/package/alerts/alert_dolphin_scheduler_status.py",
      "parameters": [

        {
          "name": "connection.timeout",
          "display_name": "Connection Timeout",
          "value": 5.0,
          "type": "NUMERIC",
          "description": "The maximum time before this alert is considered to be CRITICAL",
          "units": "seconds",
          "threshold": "CRITICAL"
        },
        {
          "name": "alertName",
          "display_name": "alertName",
          "value": "DOLPHIN_MASTER",
          "type": "STRING",
          "description": "alert name"
        }
      ]
    }
  }
],
"DOLPHIN_WORKER": [
  {
    "name": "DOLPHIN_WORKER_CHECK",
    "label": "check dolphin scheduler worker status",
    "description": "",
    "interval":10,
    "scope": "HOST",
    "enabled": true,
    "source": {
      "type": "SCRIPT",
      "path": "DOLPHIN/1.3.3/package/alerts/alert_dolphin_scheduler_status.py",
      "parameters": [

        {
          "name": "connection.timeout",
          "display_name": "Connection Timeout",
          "value": 5.0,
          "type": "NUMERIC",
          "description": "The maximum time before this alert is considered to be CRITICAL",
          "units": "seconds",
          "threshold": "CRITICAL"
        },
        {
          "name": "alertName",
          "display_name": "alertName",
          "value": "DOLPHIN_WORKER",
          "type": "STRING",
          "description": "alert name"
        }
      ]
    }
  }
],
"DOLPHIN_ALERT": [
  {
    "name": "DOLPHIN_DOLPHIN_ALERT_CHECK",
    "label": "check dolphin scheduler alert status",
    "description": "",
    "interval":10,
    "scope": "HOST",
    "enabled": true,
    "source": {
      "type": "SCRIPT",
      "path": "DOLPHIN/1.3.3/package/alerts/alert_dolphin_scheduler_status.py",
      "parameters": [

        {
          "name": "connection.timeout",
          "display_name": "Connection Timeout",
          "value": 5.0,
          "type": "NUMERIC",
          "description": "The maximum time before this alert is considered to be CRITICAL",
          "units": "seconds",
          "threshold": "CRITICAL"
        },
        {
          "name": "alertName",
          "display_name": "alertName",
          "value": "DOLPHIN_ALERT",
          "type": "STRING",
          "description": "alert name"
        }
      ]
    }
  }
]

}
}

2.4 templates
params.py脚步是对xml中界面输入的变量进行处理，定义map数据结构对数据进行处理，比如说拿到ambari部署的内置的zk hosts，比如说拿到resource mangaer hosts。比如说如果是mysql则进行一些固定信息的设置等等。定义完map数据结果，使用 .j2 默认输出key,value。再依赖于dolpin_env.py脚步将rpm包或者自己程序中的配置文件给替换了

import sys
from resource_management import *
from resource_management.core.logger import Logger
from resource_management.libraries.functions import default

Logger.initialize_logger()
reload(sys)
sys.setdefaultencoding(‘utf-8’)

#server configurations
config = Script.get_config()

conf_dir = “/etc/”
dolphin_home = “/opt/soft/dolphinscheduler”
dolphin_conf_dir = dolphin_home + “/conf”
dolphin_log_dir = dolphin_home + “/logs”
dolphin_bin_dir = dolphin_home + “/bin”
dolphin_lib_jars = dolphin_home + “/lib/*”
dolphin_pidfile_dir = “/opt/soft/run/dolphinscheduler”

rmHosts = default("/clusterHostInfo/rm_host", [])

#dolphin-env
dolphin_env_map = {}
dolphin_env_map.update(config[‘configurations’][‘dolphin-env’])

#which user to install and admin dolphin scheduler
dolphin_user = dolphin_env_map[‘dolphin.user’]
dolphin_group = dolphin_env_map[‘dolphin.group’]

#.dolphinscheduler_env.sh
dolphin_env_path = dolphin_conf_dir + ‘/env/dolphinscheduler_env.sh’
dolphin_env_content = dolphin_env_map[‘dolphinscheduler-env-content’]

#database config
dolphin_database_config = {}
dolphin_database_config[‘dolphin_database_type’] = dolphin_env_map[‘dolphin.database.type’]
dolphin_database_config[‘dolphin_database_username’] = dolphin_env_map[‘dolphin.database.username’]
dolphin_database_config[‘dolphin_database_password’] = dolphin_env_map[‘dolphin.database.password’]
if ‘mysql’ == dolphin_database_config[‘dolphin_database_type’]:
dolphin_database_config[‘dolphin_database_driver’] = ‘com.mysql.jdbc.Driver’
dolphin_database_config[‘driverDelegateClass’] = ‘org.quartz.impl.jdbcjobstore.StdJDBCDelegate’
dolphin_database_config[‘dolphin_database_url’] = ‘jdbc:mysql://’ + dolphin_env_map[‘dolphin.database.host’]
+ ‘:’ + dolphin_env_map[‘dolphin.database.port’]
+ ‘/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8’
else:
dolphin_database_config[‘dolphin_database_driver’] = ‘org.postgresql.Driver’
dolphin_database_config[‘driverDelegateClass’] = ‘org.quartz.impl.jdbcjobstore.PostgreSQLDelegate’
dolphin_database_config[‘dolphin_database_url’] = ‘jdbc:postgresql://’ + dolphin_env_map[‘dolphin.database.host’]
+ ‘:’ + dolphin_env_map[‘dolphin.database.port’]
+ ‘/dolphinscheduler’

#application-alert.properties
dolphin_alert_map = {}
wechat_push_url = ‘https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token=$toke{n}^{\prime }wecha{t}_{t}oke{n}_{u}rl{=}^{\prime }https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid=$corpId&corpsecret=KaTeX parse error: Can't use function '\"' in math mode at position 34: …m_send_msg = '{\̲"̲toparty\":\"toParty",“agentid”:“KaTeX parse error: Can't use function '\"' in math mode at position 8: agentId\̲"̲,\"msgtype\":\"…msg”},“safe”:“0”}’
wechat_user_send_msg = ‘{“touser”:“KaTeX parse error: Can't use function '\"' in math mode at position 7: toUser\̲"̲,\"agentid\":\"agentId”,“msgtype”:“markdown”,“markdown”:{“content”:"$msg"}}’

dolphin_alert_config_map = config[‘configurations’][‘dolphin-alert’]

if dolphin_alert_config_map[‘enterprise.wechat.enable’]:
dolphin_alert_map[‘enterprise.wechat.push.ur’] = wechat_push_url
dolphin_alert_map[‘enterprise.wechat.token.url’] = wechat_token_url
dolphin_alert_map[‘enterprise.wechat.team.send.msg’] = wechat_team_send_msg
dolphin_alert_map[‘enterprise.wechat.user.send.msg’] = wechat_user_send_msg

dolphin_alert_map.update(dolphin_alert_config_map)

#application-api.properties
dolphin_app_api_map = {}
dolphin_app_api_map.update(config[‘configurations’][‘dolphin-application-api’])

#common.properties
dolphin_common_map = {}

if ‘yarn-site’ in config[‘configurations’] and
‘yarn.resourcemanager.webapp.address’ in config[‘configurations’][‘yarn-site’]:
yarn_resourcemanager_webapp_address = config[‘configurations’][‘yarn-site’][‘yarn.resourcemanager.webapp.address’]
yarn_application_status_address = ‘http://’ + yarn_resourcemanager_webapp_address + ‘/ws/v1/cluster/apps/%s’
dolphin_common_map[‘yarn.application.status.address’] = yarn_application_status_address

rmHosts = default("/clusterHostInfo/rm_host", [])
if len(rmHosts) > 1:
dolphin_common_map[‘yarn.resourcemanager.ha.rm.ids’] = ‘,’.join(rmHosts)
else:
dolphin_common_map[‘yarn.resourcemanager.ha.rm.ids’] = ‘’

dolphin_common_map_tmp = config[‘configurations’][‘dolphin-common’]
data_basedir_path = dolphin_common_map_tmp[‘data.basedir.path’]
dolphin_common_map[‘dolphinscheduler.env.path’] = dolphin_env_path
dolphin_common_map.update(config[‘configurations’][‘dolphin-common’])

#datasource.properties
dolphin_datasource_map = {}
dolphin_datasource_map[‘spring.datasource.type’] = ‘com.alibaba.druid.pool.DruidDataSource’
dolphin_datasource_map[‘spring.datasource.driver-class-name’] = dolphin_database_confi[‘dolphin_database_driver’]
dolphin_datasource_map[‘spring.datasource.url’] = dolphin_database_config[‘dolphin_database_url’]
dolphin_datasource_map[‘spring.datasource.username’] = dolphin_database_config[‘dolphin_database_username’]
dolphin_datasource_map[‘spring.datasource.password’] = dolphin_database_config[‘dolphin_database_password’]
dolphin_datasource_map.update(config[‘configurations’][‘dolphin-datasource’])

#master.properties
dolphin_master_map = config[‘configurations’][‘dolphin-master’]

#quartz.properties
dolphin_quartz_map = {}
dolphin_quartz_map[‘org.quartz.jobStore.driverDelegateClass’] = dolphin_database_config[‘driverDelegateClass’]
dolphin_quartz_map.update(config[‘configurations’][‘dolphin-quartz’])

#worker.properties
dolphin_worker_map = config[‘configurations’][‘dolphin-worker’]

#zookeeper.properties
#定义各种的数据结构，dolphin_zookeeper_map，然后对其进行数据的重新赋值
#这里是需要注意的，zookeeperHosts = default("/clusterHostInfo/zookeeper_server_hosts", [])，不同的ambari版本，系统参数是不一样的，最好是打印一下
#default("/clusterHostInfo", []) 看内置参数
dolphin_zookeeper_map={}
zookeeperHosts = default("/clusterHostInfo/zookeeper_hosts", [])
if len(zookeeperHosts) > 0 and “clientPort” in config[‘configurations’][‘zoo.cfg’]:
clientPort = config[‘configurations’][‘zoo.cfg’][‘clientPort’]
zookeeperPort = “:” + clientPort + “,”
dolphin_zookeeper_map[‘zookeeper.quorum’] = zookeeperPort.join(zookeeperHosts) + “:” + clientPort
dolphin_zookeeper_map.update(config[‘configurations’][‘dolphin-zookeeper’])
if ‘spring.servlet.multipart.max-file-size’ in dolphin_app_api_map:
file_size = dolphin_app_api_map[‘spring.servlet.multipart.max-file-size’]
dolphin_app_api_map[‘spring.servlet.multipart.max-file-size’] = file_size + “MB”
if ‘spring.servlet.multipart.max-request-size’ in dolphin_app_api_map:
request_size = dolphin_app_api_map[‘spring.servlet.multipart.max-request-size’]
dolphin_app_api_map[‘spring.servlet.multipart.max-request-size’] = request_size + “MB”

最后，变量准备好了，就调用 .j2 模版将变量输出，使用 dolpin_env.py 进行配置文件的替换

2.5 theme.json，用来描述安装的页面，SETTINGS描述，如下：

{
“name”: “default”,
“description”: “Default theme for Dolphin Scheduler service”,
“configuration”: {
“layouts”: [ // 布局
{
“name”: “default”,
“tabs”: [
{
“name”: “settings”,
“display-name”: “Settings”, // 这是ambari页面显示的标签
“layout”: {
“tab-rows”: “3”, //页面有三行
“tab-columns”: “3”, //页面有三列
“sections”: [
{
“name”: “dolphin-env-config”,
“display-name”: “Dolphin Env Config”,
“row-index”: “0”, //这是第一行，之后以此增加
“column-index”: “0”, //这个不变
“row-span”: “1”,// 这个是制定的行的span，不用变
“column-span”: “2”,// 这个是制定的是列的span，不用变
“section-rows”: “1”,//一行
“section-columns”: “2”,//section有两列
“subsections”: [
{
“name”: “env-row1-col1”,
“display-name”: “Deploy User Info”,
“row-index”: “0”,
“column-index”: “0”,
“row-span”: “1”,
“column-span”: “1”
},
{
“name”: “env-row1-col2”,
“display-name”: “System Env Optimization”,
“row-index”: “0”,
“column-index”: “1”,
“row-span”: “1”,
“column-span”: “1”
}
]
},
{
“name”: “dolphin-database-config”,
“display-name”: “Database Config”,
“row-index”: “1”,// 这里是第二行
“column-index”: “0”,
“row-span”: “1”,
“column-span”: “2”,
“section-rows”: “1”,// 一行
“section-columns”: “3”,// 三列
“subsections”: [
{
“name”: “database-row1-col1”,
“row-index”: “0”,
“column-index”: “0”,
“row-span”: “1”,
“column-span”: “1”
},
{
“name”: “database-row1-col2”,
“row-index”: “0”,
“column-index”: “1”,
“row-span”: “1”,
“column-span”: “1”
},
{
“name”: “database-row1-col3”,
“row-index”: “0”,
“column-index”: “2”,
“row-span”: “1”,
“column-span”: “1”
}
]
},
{
“name”: “dynamic-config”,
“row-index”: “2”,//这里是第三行
“column-index”: “0”,
“row-span”: “1”,
“column-span”: “2”,
“section-rows”: “1”,//一行
“section-columns”: “3”,//三列
“subsections”: [
{
“name”: “dynamic-row1-col1”,
“display-name”: “Resource FS Config”,
“row-index”: “0”,
“column-index”: “0”,
“row-span”: “1”,
“column-span”: “1”
},
{
“name”: “dynamic-row1-col2”,
“display-name”: “Kerberos Info”,
“row-index”: “0”,
“column-index”: “1”,
“row-span”: “1”,
“column-span”: “1”
},
{
“name”: “dynamic-row1-col3”,
“display-name”: “Wechat Info”,
“row-index”: “0”,
“column-index”: “1”,
“row-span”: “1”,
“column-span”: “1”
}
]
}
]
}
}
]
}
],
“placement”: { //放置
“configuration-layout”: “default”,
“configs”: [
{
“config”: “dolphin-env/dolphin.database.type”,
“subsection-name”: “database-row1-col1”
},
{
“config”: “dolphin-env/dolphin.database.host”,
“subsection-name”: “database-row1-col2”
},
{
“config”: “dolphin-env/dolphin.database.port”,
“subsection-name”: “database-row1-col2”
},
{
“config”: “dolphin-env/dolphin.database.username”,
“subsection-name”: “database-row1-col3”
},
{
“config”: “dolphin-env/dolphin.database.password”,
“subsection-name”: “database-row1-col3”
},
{
“config”: “dolphin-env/dolphin.user”,
“subsection-name”: “env-row1-col1” // 这里是占位符，这里是占位符，对应的xml中的变量
},
{
“config”: “dolphin-env/dolphin.group”, // 这里是占位符，对应的xml中的变量
“subsection-name”: “env-row1-col1”
},
{
“config”: “dolphin-env/dolphinscheduler-env-content”,
“subsection-name”: “env-row1-col2”
},
{
“config”: “dolphin-common/resource.storage.type”, // 都是根据资源类型尽心判断显示的，这个在xml中对应的是列表
“subsection-name”: “dynamic-row1-col1”
},
{
“config”: “dolphin-common/resource.upload.path”, // 对于资源上传路径根据存储类型进行显示，如果HDFS和S3则显示，否则不显示
“subsection-name”: “dynamic-row1-col1”,
“depends-on”: [
{
“configs”:[
“dolphin-common/resource.storage.type” // 根据资源类型进行判断显示与否
],
“if”: “${dolphin-common/resource.storage.type} === HDFS || KaTeX parse error: Expected 'EOF', got '}' at position 341: … } }̲ ] …{dolphin-common/resource.storage.type} === HDFS”,
“then”: {
“property_value_attributes”: {
“visible”: true
}
},
“else”: {
“property_value_attributes”: {
“visible”: false
}
}
}
]
},
{
“config”: “dolphin-common/data.store2hdfs.basepath”,
“subsection-name”: “dynamic-row1-col1”,
“depends-on”: [
{
“configs”:[
“dolphin-common/resource.storage.type”
],
“if”: “KaTeX parse error: Expected 'EOF', got '}' at position 343: … } }̲ ] …{dolphin-common/resource.storage.type} === HDFS”,
“then”: {
“property_value_attributes”: {
“visible”: true
}
},
“else”: {
“property_value_attributes”: {
“visible”: false
}
}
}
]
},
{
“config”: “dolphin-common/fs.s3a.endpoint”,
“subsection-name”: “dynamic-row1-col1”,
“depends-on”: [
{
“configs”:[
“dolphin-common/resource.storage.type”
],
“if”: “KaTeX parse error: Expected 'EOF', got '}' at position 341: … } }̲ ] …{dolphin-common/resource.storage.type} === S3”,
“then”: {
“property_value_attributes”: {
“visible”: true
}
},
“else”: {
“property_value_attributes”: {
“visible”: false
}
}
}
]
},
{
“config”: “dolphin-common/fs.s3a.secret.key”,
“subsection-name”: “dynamic-row1-col1”,
“depends-on”: [
{
“configs”:[
“dolphin-common/resource.storage.type”
],
“if”: “KaTeX parse error: Expected 'EOF', got '}' at position 341: … } }̲ ] …{dolphin-common/hadoop.security.authentication.startup.state}”,
“then”: {
“property_value_attributes”: {
“visible”: true
}
},
“else”: {
“property_value_attributes”: {
“visible”: false
}
}
}
]
},
{
“config”: “dolphin-common/login.user.keytab.username”,
“subsection-name”: “dynamic-row1-col2”,
“depends-on”: [
{
“configs”:[
“dolphin-common/hadoop.security.authentication.startup.state”
],
“if”: “KaTeX parse error: Expected 'EOF', got '}' at position 357: … } }̲ ] …{dolphin-common/hadoop.security.authentication.startup.state}”,
“then”: {
“property_value_attributes”: {
“visible”: true
}
},
“else”: {
“property_value_attributes”: {
“visible”: false
}
}
}
]
},
{
“config”: “dolphin-common/kerberos.expire.time”,
“subsection-name”: “dynamic-row1-col2”,
“depends-on”: [
{
“configs”:[
“dolphin-common/hadoop.security.authentication.startup.state”
],
“if”: “KaTeX parse error: Expected 'EOF', got '}' at position 357: … } }̲ ] …{dolphin-alert/enterprise.wechat.enable}”,
“then”: {
“property_value_attributes”: {
“visible”: true
}
},
“else”: {
“property_value_attributes”: {
“visible”: false
}
}
}
]
},
{
“config”: “dolphin-alert/enterprise.wechat.secret”,
“subsection-name”: “dynamic-row1-col3”,
“depends-on”: [
{
“configs”:[
“dolphin-alert/enterprise.wechat.enable”
],
“if”: “KaTeX parse error: Expected 'EOF', got '}' at position 336: … } }̲ ] …{dolphin-alert/enterprise.wechat.enable}”,
“then”: {
“property_value_attributes”: {
“visible”: true
}
},
“else”: {
“property_value_attributes”: {
“visible”: false
}
}
}
]
},
{
“config”: “dolphin-alert/enterprise.wechat.users”,
“subsection-name”: “dynamic-row1-col3”,
“depends-on”: [
{
“configs”:[
“dolphin-alert/enterprise.wechat.enable”
],
“if”: “${dolphin-alert/enterprise.wechat.enable}”,
“then”: {
“property_value_attributes”: {
“visible”: true
}
},
“else”: {
“property_value_attributes”: {
“visible”: false
}
}
}
]
}
]
},
“widgets”: [ // 部件
{
“config”: “dolphin-env/dolphin.database.type”,
“widget”: {
“type”: “combo” // 下拉
}
},
{
“config”: “dolphin-env/dolphin.database.host”,
“widget”: {
“type”: “text-field” // 文本域
}
},
{
“config”: “dolphin-env/dolphin.database.port”,
“widget”: {
“type”: “text-field”,
“units”: [
{
“unit-name”: “int” // 设置类型
}
]
}
},
{
“config”: “dolphin-env/dolphin.database.username”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-env/dolphin.database.password”,
“widget”: {
“type”: “password”
}
},
{
“config”: “dolphin-env/dolphin.user”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-env/dolphin.group”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-env/dolphinscheduler-env-content”,
“widget”: {
“type”: “text-area” // 区域
}
},
{
“config”: “dolphin-common/resource.storage.type”,
“widget”: {
“type”: “combo”
}
},
{
“config”: “dolphin-common/resource.upload.path”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/hdfs.root.user”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/data.store2hdfs.basepath”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/fs.defaultFS”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/fs.s3a.endpoint”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/fs.s3a.access.key”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/fs.s3a.secret.key”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/hadoop.security.authentication.startup.state”,
“widget”: {
“type”: “toggle” // 就是ture/false的那种
}
},
{
“config”: “dolphin-common/java.security.krb5.conf.path”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/login.user.keytab.username”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/login.user.keytab.path”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-common/kerberos.expire.time”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-alert/enterprise.wechat.enable”,
“widget”: {
“type”: “toggle”
}
},
{
“config”: “dolphin-alert/enterprise.wechat.corp.id”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-alert/enterprise.wechat.secret”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-alert/enterprise.wechat.agent.id”,
“widget”: {
“type”: “text-field”
}
},
{
“config”: “dolphin-alert/enterprise.wechat.users”,
“widget”: {
“type”: “text-field”
}
}
]
}
}

dolphin_xxx_service.py
import time
from resource_management import *

from dolphin_env import dolphin_env

class DolphinApiService(Script):
def install(self, env):
import params
env.set_params(params)
self.install_packages(env)
Execute((‘chmod’, ‘-R’, ‘777’, params.dolphin_home))
Execute((‘chown’, ‘-R’, params.dolphin_user + “:” + params.dolphin_group, params.dolphin_home))

def configure(self, env):
    import params
    params.pika_slave = True
    env.set_params(params)
	
	# 替换配置文件
    dolphin_env()

# 启动
def start(self, env):
    import params
    env.set_params(params)
    self.configure(env)

    #init
    init_cmd=format("sh " + params.dolphin_home + "/script/create-dolphinscheduler.sh")
    Execute(init_cmd, user=params.dolphin_user)

    #upgrade
    upgrade_cmd=format("sh " + params.dolphin_home + "/script/upgrade-dolphinscheduler.sh")
    Execute(upgrade_cmd, user=params.dolphin_user)

    no_op_test = format("ls {dolphin_pidfile_dir}/api-server.pid >/dev/null 2>&1 && ps `cat {dolphin_pidfile_dir}/api-server.pid` | grep `cat {dolphin_pidfile_dir}/api-server.pid` >/dev/null 2>&1")

    start_cmd = format("sh " + params.dolphin_bin_dir + "/dolphinscheduler-daemon.sh start api-server")
    Execute(start_cmd, user=params.dolphin_user, not_if=no_op_test)

# 停止
def stop(self, env):
    import params
    env.set_params(params)
    stop_cmd = format("sh " + params.dolphin_bin_dir + "/dolphinscheduler-daemon.sh stop api-server")
    Execute(stop_cmd, user=params.dolphin_user)
    time.sleep(5)

# 检查状态
def status(self, env):
    import status_params
    env.set_params(status_params)
    check_process_status(status_params.dolphin_run_dir + "api-server.pid")

if name == “main”:
DolphinApiService().execute()