dolphinscheduler 1.3.6 集群部署

本文档详细介绍了如何使用root用户进行DolphinScheduler的分布式部署,包括环境准备、免密配置、服务器列表设定、解压安装、数据库配置、环境变量设置、安装配置脚本以及启动脚本的执行。此外,还提到了DolphinScheduler在Zookeeper中显示数据问题的解决办法。整个过程旨在简化多用户部署的复杂性,实现一键部署。
摘要由CSDN通过智能技术生成

大家 基本 用添加 dolphinscheduler 或者ds 用户的方式去部署,每台都是需要添加用户.很麻烦. 这里介绍root 部署.

1. 准备环境: zookeeper 3.6.3环境准备 jdk1.8 配置环境

2. 免密配置

ssh-keygen -t rsa   三台
ssh-copy-id hdp1 三台
scp /root/.ssh/authorized_keys hdp2:/root/.ssh
scp /root/.ssh/authorized_keys hdp3:/root/.ssh

3. 服务器列表(内存尽量大些 建议 8GB+)

节点HOSTNAME服务
192.168.174.204hdp1MasterServer WorkerServer ApiApplicationServer AlertServer LoggerServer
192.168.174.205hdp2MasterServer WorkerServer LoggerServer
192.168.174.206hdp3WorkerServer LoggerServer

4.解压 安装包

tar -zxvf apache-dolphinscheduler-1.3.6-bin.tar.gz

5.配置数据库

CREATE USER  dolphinscheduler  IDENTIFIED BY 'dolphinscheduler';
CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'%' IDENTIFIED BY 'dolphinscheduler';

/opt/soft/dolphin/conf/datasource.properties

# mysql
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://192.168.*.*:3306/dolphinscheduler?characterEncoding=UTF-8&allowMultiQueries=true
spring.datasource.username=dolphinscheduler
spring.datasource.password=dolphinscheduler

执行数据库脚本 sh create-dolphinscheduler.sh

6.配置环境

install_config.conf

dbtype="mysql"
# db config
# db address and port
dbhost="192.168.*.*:3306"
# db username
username="dolphinscheduler"
# database name
dbname="dolphinscheduler"
# db passwprd
# NOTICE: if there are special characters, please use the \ to escape, for example, `[` escape to `\[`
password="dolphinscheduler"
# zk cluster
zkQuorum="hdp1:2181,hdp2:2181,hdp3:2181"
# Note: the target installation path for dolphinscheduler, please not config as the same as the current path (pwd)
installPath="/usr/local/dolphinscheduler"
# deployment user
# Note: the deployment user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled, the root directory needs to be created by itself
deployUser="root"
# resource storage type: HDFS, S3, NONE
resourceStorageType="HDFS"
defaultFS="hdfs://hdp3:9000"
singleYarnIp="hdp3"
resourceUploadPath="/dolphinscheduler"
hdfsRootUser="root"
# api server port
apiServerPort="12345"
# install hosts
# Note: install the scheduled hostname list. If it is pseudo-distributed, just write a pseudo-distributed hostname
ips="hdp1,hdp2,hdp3"
# run master machine
# Note: list of hosts hostname for deploying master
masters="hdp1,hdp2"
# run worker machine
# note: need to write the worker group name of each worker, the default value is "default"
workers="hdp1:default,hdp2:default,hdp3:default"
# run alert machine
# note: list of machine hostnames for deploying alert server
alertServer="hdp1"
# run api machine
# note: list of machine hostnames for deploying api server
apiServers="hdp1"

7.修改 dolphinscheduler_env.sh

export HADOOP_HOME=/opt/soft/hadoop
export HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
export JAVA_HOME=/opt/jdk1.8.0_141
export PATH=$HADOOP_HOME/bin:$JAVA_HOME/bin:$PATH

8.执行 脚本

sh install

访问 hdp3:12345/dolphinscheduler

默认用户名:admin

默认密码:dolphinscheduler123

DolphinScheduler监页面里显示的Zookeeper数据全是-1

zoo.cfg文件里配置:4lw.commands.whitelist=* 然后重启 zk 服务

 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值