大家 基本 用添加 dolphinscheduler 或者ds 用户的方式去部署,每台都是需要添加用户.很麻烦. 这里介绍root 部署.
1. 准备环境: zookeeper 3.6.3环境准备 jdk1.8 配置环境
2. 免密配置
ssh-keygen -t rsa 三台
ssh-copy-id hdp1 三台
scp /root/.ssh/authorized_keys hdp2:/root/.ssh
scp /root/.ssh/authorized_keys hdp3:/root/.ssh
3. 服务器列表(内存尽量大些 建议 8GB+)
节点 | HOSTNAME | 服务 |
---|---|---|
192.168.174.204 | hdp1 | MasterServer WorkerServer ApiApplicationServer AlertServer LoggerServer |
192.168.174.205 | hdp2 | MasterServer WorkerServer LoggerServer |
192.168.174.206 | hdp3 | WorkerServer LoggerServer |
4.解压 安装包
tar -zxvf apache-dolphinscheduler-1.3.6-bin.tar.gz
5.配置数据库
CREATE USER dolphinscheduler IDENTIFIED BY 'dolphinscheduler';
CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'%' IDENTIFIED BY 'dolphinscheduler';
/opt/soft/dolphin/conf/datasource.properties
# mysql
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://192.168.*.*:3306/dolphinscheduler?characterEncoding=UTF-8&allowMultiQueries=true
spring.datasource.username=dolphinscheduler
spring.datasource.password=dolphinscheduler
执行数据库脚本 sh create-dolphinscheduler.sh
6.配置环境
install_config.conf
dbtype="mysql"
# db config
# db address and port
dbhost="192.168.*.*:3306"
# db username
username="dolphinscheduler"
# database name
dbname="dolphinscheduler"
# db passwprd
# NOTICE: if there are special characters, please use the \ to escape, for example, `[` escape to `\[`
password="dolphinscheduler"
# zk cluster
zkQuorum="hdp1:2181,hdp2:2181,hdp3:2181"
# Note: the target installation path for dolphinscheduler, please not config as the same as the current path (pwd)
installPath="/usr/local/dolphinscheduler"
# deployment user
# Note: the deployment user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled, the root directory needs to be created by itself
deployUser="root"
# resource storage type: HDFS, S3, NONE
resourceStorageType="HDFS"
defaultFS="hdfs://hdp3:9000"
singleYarnIp="hdp3"
resourceUploadPath="/dolphinscheduler"
hdfsRootUser="root"
# api server port
apiServerPort="12345"
# install hosts
# Note: install the scheduled hostname list. If it is pseudo-distributed, just write a pseudo-distributed hostname
ips="hdp1,hdp2,hdp3"
# run master machine
# Note: list of hosts hostname for deploying master
masters="hdp1,hdp2"
# run worker machine
# note: need to write the worker group name of each worker, the default value is "default"
workers="hdp1:default,hdp2:default,hdp3:default"
# run alert machine
# note: list of machine hostnames for deploying alert server
alertServer="hdp1"
# run api machine
# note: list of machine hostnames for deploying api server
apiServers="hdp1"
7.修改 dolphinscheduler_env.sh
export HADOOP_HOME=/opt/soft/hadoop
export HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
export JAVA_HOME=/opt/jdk1.8.0_141
export PATH=$HADOOP_HOME/bin:$JAVA_HOME/bin:$PATH
8.执行 脚本
sh install
访问 hdp3:12345/dolphinscheduler
默认用户名:admin
默认密码:dolphinscheduler123
DolphinScheduler监页面里显示的Zookeeper数据全是-1
zoo.cfg文件里配置:4lw.commands.whitelist=* 然后重启 zk 服务