DSS部署-12、DSS安装

已于 2022-03-07 17:46:04 修改
阅读量3.3k 收藏 1
点赞数
分类专栏： DataSphereStudio部署流程 文章标签： 人工智能 物联网 iot
于 2022-03-06 01:14:05 首次发布
沧海之巅
本文链接：https://blog.csdn.net/linjie_830914/article/details/123304520
版权

DataSphere Studio一键安装使用文档

补充nginx配置，测试yum是否好用，准备配置文件

links默认使用python，建议安装python2

web的install.sh中，需要本地安装后，然后去掉nginx的安装，及防火墙的处理部分脚本

修改全家中中conf的config.sh中的nginx端口为非8088 ，不能与yarn冲突

一、使用前环境准备

a. 基础软件安装

Linkix需要的命令工具（在正式安装前，脚本会自动检测这些命令是否可用，如果不存在会尝试自动安装，安装失败则需用户手动安装以下基础shell命令工具）：

  • telnet tar sed dos2unix mysql yum unzip expect
  • MySQL (5.5+)
  • JDK (1.8.0_141以上)
  • Python(2.7)
  • Nginx
  • Hadoop(2.7.2，Hadoop其他版本需自行编译Linkis) ，安装的机器必须支持执行 hdfs dfs -ls / 命令
  • Hive(2.3.3，Hive其他版本需自行编译Linkis)，安装的机器必须支持执行hive -e "show databases"命令
  • Spark(支持2.0以上所有版本) ，安装的机器必须支持执行spark-sql -e "show databases"命令

Tips:

如您是第一次安装Hadoop，单机部署Hadoop可参考：Hadoop单机部署 ；分布式部署Hadoop可参考：Hadoop分布式部署。

如您是第一次安装Hive，可参考：Hive快速安装部署。

如您是第一次安装Spark，On Yarn模式可参考：Spark on Yarn部署。

b. 创建用户

​ 例如: 部署用户是hadoop账号（可以不是hadoop用户，但是推荐使用Hadoop的超级用户进行部署，这里只是一个示例）

  1. 在所有需要部署的机器上创建部署用户，用于安装
sudo useradd hadoop
  1. 因为Linkis的服务是以 sudo -u ${linux-user} 方式来切换引擎，从而执行作业，所以部署用户需要有 sudo 权限，而且是免密的。
vim /etc/sudoers
hadoop  ALL=(ALL)       NOPASSWD: NOPASSWD: ALL
  1. 确保部署 DSS 和 Linkis 的服务器可正常执行 hdfs 、hive -e 和 spark-sql -e 等命令。在一键安装脚本中，会对组件进行检查。
  2. 如果您的Pyspark想拥有画图功能，则还需在所有安装节点，安装画图模块。命令如下：
下载pip（适用于python3）
 curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
 
 下载pip（适用于python2）
 curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py
 安装pip
 python get-pip.py
 升级pip
 pip install --upgrade pip
 安装matplotlib
 python -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple matplotlib
 pip install matplotlib

c.安装准备

自行编译或者去组件release页面下载安装包：

  1. 下载安装包
  1. 下载 DSS & LINKIS 一键安装部署包，并解压。以下是一键安装部署包的层级目录结构：
├── dss_linkis # 一键部署主目录
 ├── bin # 用于一键安装，以及一键启动 DSS + Linkis
 ├── conf # 一键部署的参数配置目录
 ├── wedatasphere-dss-x.x.x-dist.tar.gz # DSS后台安装包
 ├── wedatasphere-dss-web-x.x.x-dist.zip # DSS前端安装包
 ├── wedatasphere-linkis-x.x.x-dist.tar.gz # Linkis安装包

d. 修改配置

​ 打开conf/config.sh，按需修改相关配置参数：

vim conf/config.sh

参数说明如下：

说明，DSS_WEB_PORT端口需要保证不能 与YARN REST URL端口冲突，可以改为8099或其他可用端口

#################### 一键安装部署的基本配置 ####################

# 部署用户，默认为当前登录用户，非必须不建议修改
# deployUser=hadoop

# 非必须不建议修改
# LINKIS_VERSION=1.0.2

### DSS Web，本机安装无需修改
#DSS_NGINX_IP=127.0.0.1
#DSS_WEB_PORT=8099

# 非必须不建议修改
#DSS_VERSION=1.0.0

## Java应用的堆栈大小。如果部署机器的内存少于8G，推荐128M；达到16G时，推荐至少256M；如果想拥有非常良好的用户使用体验，推荐部署机器的内存至少达到32G。
export SERVER_HEAP_SIZE="128M"

############################################################
##################### Linkis 的配置开始 #####################
########### 非注释的参数必须配置，注释掉的参数可按需修改 ##########
############################################################

### DSS工作空间目录
WORKSPACE_USER_ROOT_PATH=file:///tmp/linkis/
### 用户 HDFS 根路径
HDFS_USER_ROOT_PATH=hdfs:///tmp/linkis
### 结果集路径: file 或者 hdfs path
RESULT_SET_ROOT_PATH=hdfs:///tmp/linkis

### Path to store started engines and engine logs, must be local
ENGINECONN_ROOT_PATH=/appcom/tmp

#ENTRANCE_CONFIG_LOG_PATH=hdfs:///tmp/linkis/

### ==HADOOP配置文件路径，必须配置==
HADOOP_CONF_DIR=/appcom/config/hadoop-config
### HIVE CONF DIR
HIVE_CONF_DIR=/appcom/config/hive-config
### SPARK CONF DIR
SPARK_CONF_DIR=/appcom/config/spark-config

# for install
#LINKIS_PUBLIC_MODULE=lib/linkis-commons/public-module


## YARN REST URL
YARN_RESTFUL_URL=http://127.0.0.1:8088

## Engine版本配置，不配置则采用默认配置
#SPARK_VERSION
#SPARK_VERSION=2.4.3
##HIVE_VERSION
#HIVE_VERSION=1.2.1
#PYTHON_VERSION=python2

## LDAP is for enterprise authorization, if you just want to have a try, ignore it.
#LDAP_URL=ldap://localhost:1389/
#LDAP_BASEDN=dc=webank,dc=com
#LDAP_USER_NAME_FORMAT=cn=%s@xxx.com,OU=xxx,DC=xxx,DC=com

# Microservices Service Registration Discovery Center
#LINKIS_EUREKA_INSTALL_IP=127.0.0.1         
#LINKIS_EUREKA_PORT=20303
#LINKIS_EUREKA_PREFER_IP=true

###  Gateway install information
#LINKIS_GATEWAY_PORT =127.0.0.1
#LINKIS_GATEWAY_PORT=9001

### ApplicationManager
#LINKIS_MANAGER_INSTALL_IP=127.0.0.1
#LINKIS_MANAGER_PORT=9101

### EngineManager
#LINKIS_ENGINECONNMANAGER_INSTALL_IP=127.0.0.1
#LINKIS_ENGINECONNMANAGER_PORT=9102

### EnginePluginServer
#LINKIS_ENGINECONN_PLUGIN_SERVER_INSTALL_IP=127.0.0.1
#LINKIS_ENGINECONN_PLUGIN_SERVER_PORT=9103

### LinkisEntrance
#LINKIS_ENTRANCE_INSTALL_IP=127.0.0.1
#LINKIS_ENTRANCE_PORT=9104

###  publicservice
#LINKIS_PUBLICSERVICE_INSTALL_IP=127.0.0.1
#LINKIS_PUBLICSERVICE_PORT=9105

### cs
#LINKIS_CS_INSTALL_IP=127.0.0.1
#LINKIS_CS_PORT=9108

##################### Linkis 的配置完毕 #####################

############################################################
####################### DSS 的配置开始 #######################
########### 非注释的参数必须配置，注释掉的参数可按需修改 ##########
############################################################

# 用于存储发布到 Schedulis 的临时ZIP包文件
WDS_SCHEDULER_PATH=file:///appcom/tmp/wds/scheduler

### This service is used to provide dss-framework-project-server capability.
#DSS_FRAMEWORK_PROJECT_SERVER_INSTALL_IP=127.0.0.1
#DSS_FRAMEWORK_PROJECT_SERVER_PORT=9002

### This service is used to provide dss-framework-orchestrator-server capability.
#DSS_FRAMEWORK_ORCHESTRATOR_SERVER_INSTALL_IP=127.0.0.1
#DSS_FRAMEWORK_ORCHESTRATOR_SERVER_PORT=9003

### This service is used to provide dss-apiservice-server capability.
#DSS_APISERVICE_SERVER_INSTALL_IP=127.0.0.1
#DSS_APISERVICE_SERVER_PORT=9004

### This service is used to provide dss-workflow-server capability.
#DSS_WORKFLOW_SERVER_INSTALL_IP=127.0.0.1
#DSS_WORKFLOW_SERVER_PORT=9005

### dss-flow-Execution-Entrance
### This service is used to provide flow execution capability.
#DSS_FLOW_EXECUTION_SERVER_INSTALL_IP=127.0.0.1
#DSS_FLOW_EXECUTION_SERVER_PORT=9006

### This service is used to provide dss-datapipe-server capability.
#DSS_DATAPIPE_SERVER_INSTALL_IP=127.0.0.1
#DSS_DATAPIPE_SERVER_PORT=9008

##sendemail配置，只影响DSS工作流中发邮件功能
EMAIL_HOST=smtp.163.com
EMAIL_PORT=25
EMAIL_USERNAME=xxx@163.com
EMAIL_PASSWORD=xxxxx
EMAIL_PROTOCOL=smtp 
####################### DSS 的配置结束 #######################

如下地址需要配置

###HADOOP CONF DIR #/appcom/config/hadoop-config
HADOOP_CONF_DIR=/opt/modules/hadoop-2.7.2/etc/hadoop/
###HIVE CONF DIR #/appcom/config/hive-config
HIVE_CONF_DIR=/opt/modules/apache-hive-2.3.3/conf
###SPARK CONF DIR #/appcom/config/spark-config
SPARK_CONF_DIR=/opt/modules/spark-2.3.2-bin-hadoop2.7/conf

e. 修改数据库配置

请确保配置的数据库，安装机器可以正常访问，否则将会出现DDL和DML导入失败的错误。

vi conf/db.sh 
### 配置DSS数据库
MYSQL_HOST=127.0.0.1
MYSQL_PORT=3306
MYSQL_DB=dss
MYSQL_USER=root
MYSQL_PASSWORD=asdf1234

## Hive metastore的数据库配置，用于Linkis访问Hive的元数据信息
HIVE_HOST=127.0.0.1
HIVE_PORT=3306
HIVE_DB=hive
HIVE_USER=root
HIVE_PASSWORD=asdf1234

f.修改wedatasphere-dss-web-1.0.1-dist配置

install.sh中的如下部分需要处理

centos7(){
    # nginx是否安装
    #sudo rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
    #调整点：1
    # yum安装的部分nginx，缺少“/etc/nginx/conf.d/文件夹”，所以nginx手动安装，详见第8部分
    #sudo yum install -y nginx 
    #echo "Nginx installed successfully"

    # 配置nginx
    dssConf

    # 解决 0.0.0.0:8888 问题
    yum -y install policycoreutils-python
    semanage port -a -t http_port_t -p tcp $dss_port

    # 开放前端访问端口
    #调整点2
    #【如果用于测试，本地已关闭防火墙，不需要执行】
    #firewall-cmd --zone=public --add-port=$dss_port/tcp --permanent

    #调整点3
    #重启防火墙
    # 【如果用于测试，本地已关闭防火墙，不需要执行】
    #firewall-cmd --reload

    # 启动nginx
    systemctl restart nginx

    # 调整SELinux的参数
    sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
    # 临时生效
    setenforce 0

}

二、安装和使用

1. 执行安装脚本：

sh bin/install.sh
# 看具体执行到哪一步
sh -v bin/install.sh

2. 安装步骤

  • 该安装脚本会检查各项集成环境命令，如果没有请按照提示进行安装，以下命令为必须项【环境准备时已经安装】

    yum java mysql zip unzip expect telnet tar sed dos2unix nginx

  • 安装时，脚本会询问您是否需要初始化数据库并导入元数据，Linkis 和 DSS 均会询问。

    第一次安装必须选是。

3. 是否安装成功：

通过查看控制台打印的日志信息查看是否安装成功。

如果有错误信息，可以查看具体报错原因。

4. 启动服务

(1) 启动服务：

在安装目录执行以下命令，启动所有服务：

sh bin/start-all.sh

如果启动产生了错误信息，可以查看具体报错原因。启动后，各项微服务都会进行通信检测，如果有异常则可以帮助用户定位异常日志和原因。

(2) 查看是否启动成功

可以在Eureka界面查看 Linkis & DSS 后台各微服务的启动情况。 Eureka

(3) 谷歌浏览器访问：

请使用谷歌浏览器访问以下前端地址：

http://DSS_NGINX_IP:DSS_WEB_PORT 启动日志会打印此访问地址。登陆时管理员的用户名和密码均为部署用户名，如部署用户为hadoop，则管理员的用户名/密码为：hadoop/hadoop。

(4) 停止服务：

在安装目录执行以下命令，停止所有服务：

sh bin/stop-all.sh

通过如下脚本判断是否停止成功，如果停止失败可以通过kill结束进程

ps -ef|grep DSS
ps -ef|grep Linkis
ps -ef|grep eureka
(5) 安装成功后，有6个DSS服务，8个Linkis服务

LINKIS-CG-ENGINECONNMANAGER
LINKIS-CG-ENGINEPLUGIN
LINKIS-CG-ENTRANCE
LINKIS-CG-LINKISMANAGER
LINKIS-MG-EUREKA
LINKIS-MG-GATEWAY
LINKIS-PS-CS
LINKIS-PS-PUBLICSERVICE

5.安装日志 install.sh

[hadoop@dss dssLinksFamilyMeals]$ sh bin/install.sh


########################################################################################
########################################################################################
Welcome to DSS & Linkis Deployment Service!
Suitable for Linkis and DSS first installation, please be sure the environment is ready.
########################################################################################
########################################################################################

It is recommended to use 5G memory.
Each service is set to 256M, with a minimum of 128M.
The default configuration is 256M.
If you need to modify it, please modify conf/config.sh

Are you sure you have installed the database?
If installed, enter 1, otherwise enter 0

Please input the choice:1

Do you need to check the installation environment?
Enter 1 if necessary, otherwise enter 0

Please input the choice:0



########################################################################
###################### Start to install Linkis #########################
########################################################################
Start to unzip linkis package.
Succeed to + Unzip linkis package to /opt/modules/dssLinksFamilyMeals/linkis-pre-install.
Start to replace linkis field value.
End to replace linkis field value.
<-----start to check used cmd---->
<-----end to check used cmd---->
Succeed to + check env
step1:load config
Succeed to + load config
Do you want to clear Linkis table information in the database?
 1: Do not execute table-building statements
 2: Dangerous! Clear all data and rebuild the tables
 other: exit

Please input the choice:2
You chose Rebuild the table
create hdfs  directory and local directory
Succeed to + create file:///tmp/linkis/ directory
Succeed to + create hdfs:///tmp/linkis directory
Succeed to + create hdfs:///tmp/linkis directory
rm: cannot remove ‘/opt/modules/dssLinksFamilyMeals/linkis-bak’: No such file or directory
mv  /opt/modules/dssLinksFamilyMeals/linkis  /opt/modules/dssLinksFamilyMeals/linkis-bak
create dir LINKIS_HOME: /opt/modules/dssLinksFamilyMeals/linkis
Succeed to + Create the dir of /opt/modules/dssLinksFamilyMeals/linkis
Start to cp /opt/modules/dssLinksFamilyMeals/linkis-pre-install/linkis-package to /opt/modules/dssLinksFamilyMeals/linkis.
Succeed to + cp /opt/modules/dssLinksFamilyMeals/linkis-pre-install/linkis-package to /opt/modules/dssLinksFamilyMeals/linkis
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
Succeed to + source linkis_ddl.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
+-----------------+
| @label_id := id |
+-----------------+
|               1 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|               2 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|               3 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|               4 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              11 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              12 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              13 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              14 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              15 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              16 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              17 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              18 |
+-----------------+
+-----------------+
| @label_id := id |
+-----------------+
|              19 |
+-----------------+
Succeed to + source linkis_dml.sql
Rebuild the table
Update config...
update conf /opt/modules/dssLinksFamilyMeals/linkis/conf/linkis.properties
update conf /opt/modules/dssLinksFamilyMeals/linkis/conf/linkis-mg-gateway.properties
update conf /opt/modules/dssLinksFamilyMeals/linkis/conf/linkis-ps-publicservice.properties
Congratulations! You have installed Linkis 1.0.3 successfully, please use sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-start-all.sh to start it!
Your default account password ishadoop/a61035488
Succeed to + install Linkis


#########################################################################
###################### Start to install DSS Service #####################
#########################################################################
Succeed to + Create the dir of /opt/modules/dssLinksFamilyMeals/dss-pre-install
Start to unzip dss server package.
Succeed to + Unzip dss server package to /opt/modules/dssLinksFamilyMeals/dss-pre-install
Start to replace dss field value.
End to replace dss field value.
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)
Succeed to + execute java --version
step1:load config
Do you want to clear Dss table information in the database?
 1: Do not execute table-building statements
 2: Dangerous! Clear all data and rebuild the tables.

Please input the choice:2
You chose Rebuild the table
Simple installation mode
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)
Succeed to + execute java --version
telnet check for your MYSQL, if you wait for a long time,may be your MYSQL does not prepared
MYSQL is OK.
mysql: [Warning] Using a password on the command line interface can be insecure.
Succeed to + source dss_ddl.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
+---------------------------------+
| @dss_appconn_orchestratorId:=id |
+---------------------------------+
|                               2 |
+---------------------------------+
+-----------------------------+
| @dss_appconn_workflowId:=id |
+-----------------------------+
|                           3 |
+-----------------------------+
+---------------------------------+
| @dss_appconn_eventcheckerId:=id |
+---------------------------------+
|                               5 |
+---------------------------------+
+--------------------------------+
| @dss_appconn_datacheckerId:=id |
+--------------------------------+
|                              6 |
+--------------------------------+
Succeed to + source dss_dml_real.sql
Rebuild the table
step2:update config
rm: cannot remove ‘/opt/modules/dssLinksFamilyMeals/dss-bak’: No such file or directory
mv  /opt/modules/dssLinksFamilyMeals/dss  /opt/modules/dssLinksFamilyMeals/dss-bak
create dir SERVER_HOME: /opt/modules/dssLinksFamilyMeals/dss
Succeed to + Create the dir of /opt/modules/dssLinksFamilyMeals/dss
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-framework-project-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-framework-orchestrator-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-apiservice-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-datapipe-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-flow-execution-server.properties
Succeed to + subsitution /opt/modules/dssLinksFamilyMeals/dss/conf/dss-workflow-server.properties
Congratulations! You have installed DSS 1.0.1 successfully, please use sbin/dss-start-all.sh to start it!
Succeed to + install DSS Service


###########################################################################
###################### Start to install DSS & Linkis Web ##################
###########################################################################
Succeed to + Create the dir of /opt/modules/dssLinksFamilyMeals/web
Start to unzip dss web package.
Succeed to + Unzip dss web package to /opt/modules/dssLinksFamilyMeals/web
Start to replace dss web field value.
End to replace  dss web field value.
dss front-end deployment script
linux
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.bupt.edu.cn
 * epel: mirror.earthlink.iq
 * extras: mirrors.cn99.com
 * updates: mirrors.cn99.com
Package policycoreutils-python-2.5-34.el7.x86_64 already installed and latest version
Nothing to do
ValueError: Port tcp/8088 already defined
Succeed to + install DSS & Linkis Web

Eureka configuration path of Linkis: linkis/conf/application-linkis.yml
Eureka configuration path of DSS : dss/conf/application-dss.yml

Congratulations! You have installed DSS & Linkis successfully, please use bin/start-all.sh to start it!

6.启动脚本 start-all.sh



[hadoop@dss dssLinksFamilyMeals]$
[hadoop@dss dssLinksFamilyMeals]$ bin/start-all.sh
########################################################################
###################### Begin to start Linkis ###########################
########################################################################
We will start all linkis applications, it will take some time, please wait
<-------------------------------->
Begin to start mg-eureka
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart mg-eureka
server mg-eureka is not running
Start to check whether the mg-eureka is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-mg-eureka
nohup: redirecting stderr to stdout
server linkis-mg-eureka start succeeded!
Succeed to + End to start mg-eureka
<-------------------------------->
<-------------------------------->
Begin to start mg-gateway
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart mg-gateway
server mg-gateway is not running
Start to check whether the mg-gateway is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-mg-gateway
nohup: redirecting stderr to stdout
server linkis-mg-gateway start succeeded!
Succeed to + End to start mg-gateway
<-------------------------------->
<-------------------------------->
Begin to start ps-publicservice
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart ps-publicservice
server ps-publicservice is not running
Start to check whether the ps-publicservice is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-ps-publicservice
nohup: redirecting stderr to stdout
server linkis-ps-publicservice start succeeded!
Succeed to + End to start ps-publicservice
<-------------------------------->
<-------------------------------->
Begin to start cg-linkismanager
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart cg-linkismanager
server cg-linkismanager is not running
Start to check whether the cg-linkismanager is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-cg-linkismanager
nohup: redirecting stderr to stdout
server linkis-cg-linkismanager start succeeded!
Succeed to + End to start cg-linkismanager
<-------------------------------->
<-------------------------------->
Begin to start ps-cs
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart ps-cs
server ps-cs is not running
Start to check whether the ps-cs is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-ps-cs
nohup: redirecting stderr to stdout
server linkis-ps-cs start succeeded!
Succeed to + End to start ps-cs
<-------------------------------->
<-------------------------------->
Begin to start cg-entrance
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart cg-entrance
server cg-entrance is not running
Start to check whether the cg-entrance is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-cg-entrance
nohup: redirecting stderr to stdout
server linkis-cg-entrance start succeeded!
Succeed to + End to start cg-entrance
<-------------------------------->
<-------------------------------->
Begin to start cg-engineconnmanager
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart cg-engineconnmanager
server cg-engineconnmanager is not running
Start to check whether the cg-engineconnmanager is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-cg-engineconnmanager
nohup: redirecting stderr to stdout
server linkis-cg-engineconnmanager start succeeded!
Succeed to + End to start cg-engineconnmanager
<-------------------------------->
<-------------------------------->
Begin to start cg-engineplugin
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh restart cg-engineplugin
server cg-engineplugin is not running
Start to check whether the cg-engineplugin is running
Start server, startup script:  /opt/modules/dssLinksFamilyMeals/linkis/sbin/ext/linkis-cg-engineplugin
nohup: redirecting stderr to stdout
server linkis-cg-engineplugin start succeeded!
Succeed to + End to start cg-engineplugin
<-------------------------------->
start-all shell script executed completely
Start to check all linkis microservice
<-------------------------------->
Begin to check mg-eureka
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status mg-eureka
12210
server mg-eureka is running.
<-------------------------------->
<-------------------------------->
Begin to check mg-gateway
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status mg-gateway
12248
server mg-gateway is running.
<-------------------------------->
<-------------------------------->
Begin to check ps-publicservice
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status ps-publicservice
12317
server ps-publicservice is running.
<-------------------------------->
<-------------------------------->
Begin to check ps-cs
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status ps-cs
12465
server ps-cs is running.
<-------------------------------->
<-------------------------------->
Begin to check cg-linkismanager
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status cg-linkismanager
12363
server cg-linkismanager is running.
<-------------------------------->
<-------------------------------->
Begin to check cg-entrance
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status cg-entrance
12519
server cg-entrance is running.
<-------------------------------->
<-------------------------------->
Begin to check cg-engineconnmanager
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status cg-engineconnmanager
12575
server cg-engineconnmanager is running.
<-------------------------------->
<-------------------------------->
Begin to check cg-engineplugin
Is local execution:sh /opt/modules/dssLinksFamilyMeals/linkis/sbin/linkis-daemon.sh status cg-engineplugin
12631
server cg-engineplugin is running.
<-------------------------------->
Linkis started successfully
Succeed to + start Linkis


########################################################################
###################### Begin to start DSS Service ######################
########################################################################
We will start all dss applications, it will take some time, please wait
<-------------------------------->
Begin to start dss-framework-project-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-framework-project-server
server dss-framework-project-server is not running
Start to check whether the dss-framework-project-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-framework-project-server
nohup: redirecting stderr to stdout
server dss-framework-project-server start succeeded!
End to start dss-framework-project-server
<-------------------------------->
<-------------------------------->
Begin to start dss-framework-orchestrator-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-framework-orchestrator-server
server dss-framework-orchestrator-server is not running
Start to check whether the dss-framework-orchestrator-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-framework-orchestrator-server
nohup: redirecting stderr to stdout
server dss-framework-orchestrator-server start succeeded!
End to start dss-framework-orchestrator-server
<-------------------------------->
<-------------------------------->
Begin to start dss-apiservice-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-apiservice-server
server dss-apiservice-server is not running
Start to check whether the dss-apiservice-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-apiservice-server
nohup: redirecting stderr to stdout
server dss-apiservice-server start succeeded!
End to start dss-apiservice-server
<-------------------------------->
<-------------------------------->
Begin to start dss-datapipe-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-datapipe-server
server dss-datapipe-server is not running
Start to check whether the dss-datapipe-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-datapipe-server
nohup: redirecting stderr to stdout
server dss-datapipe-server start succeeded!
End to start dss-datapipe-server
<-------------------------------->
<-------------------------------->
Begin to start dss-workflow-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-workflow-server
server dss-workflow-server is not running
Start to check whether the dss-workflow-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-workflow-server
nohup: redirecting stderr to stdout
server dss-workflow-server start succeeded!
End to start dss-workflow-server
<-------------------------------->
<-------------------------------->
Begin to start dss-flow-execution-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh restart dss-flow-execution-server
server dss-flow-execution-server is not running
Start to check whether the dss-flow-execution-server is running
Start to start server, startup script:  /opt/modules/dssLinksFamilyMeals/dss/sbin/ext/dss-flow-execution-server
nohup: redirecting stderr to stdout
server dss-flow-execution-server start succeeded!
End to start dss-flow-execution-server
<-------------------------------->
<-------------------------------->
Begin to check dss-framework-project-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-framework-project-server
server dss-framework-project-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-framework-orchestrator-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-framework-orchestrator-server
server dss-framework-orchestrator-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-apiservice-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-apiservice-server
server dss-apiservice-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-datapipe-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-datapipe-server
server dss-datapipe-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-workflow-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-workflow-server
server dss-workflow-server is running.
<-------------------------------->
<-------------------------------->
Begin to check dss-flow-execution-server
Is local execution:sh /opt/modules/dssLinksFamilyMeals/dss/sbin/dss-daemon.sh status dss-flow-execution-server
server dss-flow-execution-server is running.
<-------------------------------->
Succeed to + start DSS Service


########################################################################
###################### Begin to start DSS & Linkis web ##########################
########################################################################
Succeed to + start DSS & Linkis Web

===============================================
There are eight micro services in Linkis：
linkis-cg-engineconnmanager
linkis-cg-engineplugin
linkis-cg-entrance
linkis-cg-linkismanager
linkis-mg-eureka
linkis-mg-gateway
linkis-ps-cs
linkis-ps-publicservice
-----------------------------------------------
There are six micro services in DSS：
dss-framework-project-server
dss-framework-orchestrator-server-dev
dss-workflow-server-dev
dss-flow-entrance
dss-datapipe-server
dss-apiservice-server
===============================================

Log path of Linkis: linkis/logs
Log path of DSS : dss/logs

You can check DSS & Linkis by acessing eureka URL: http://192.168.122.67:20303
You can acess DSS & Linkis Web by http://192.168.122.67:8088

[hadoop@dss dssLinksFamilyMeals]$

7.日志说明

dss路径下
dss-apiservice-server.out
dss-datapipe-server.out
dss-flow-execution-server.out
dss-framework-orchestrator-server.out
dss-framework-project-server.out
dss-workflow-server.out

linkis路径下
linkis-cg-engineconnmanager.out
linkis-cg-engineplugin.out
linkis-cg-entrance.out
linkis-cg-linkismanager.out
linkis-mg-eureka.out
linkis-mg-gateway.out
linkis-ps-cs.out
linkis-ps-publicservice.out

三、相关访问地址

nginx http://192.168.122.67/

hadoop http://192.168.122.67:50070/dfshealth.html

spark http://192.168.122.67:8080/

spark http://192.168.122.67:8081/

eureka URL: http://192.168.122.67:20303

DSS & Linkis Web by http://192.168.122.67:8088

登录密码可以从日志中查到，参考如下信息

Your default account password is hadoop/5f8a94fae

沧海之巅
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
专栏目录
DH-DSS-H8900智慧园区综合管理平台
03-01
DH-DSS-H8900 系列是为通用性公共建筑物提供安全和高效管理而打造的智慧园区综合管理平台，通过融合大华在安防和智能化领域的专业经验和前沿技术，集成视频，门禁、报警，停车场，考 勤，访客，可视对讲，信息发布等...
DarwinStreamingSrvr6.0.3-Source.tar+dss-6.0.3.patch+dss-hh-20080728-1.patch
01-12
DarwinStreamingSrvr6.0.3-Source.tar+dss-6.0.3.patch+dss-hh-20080728-1.patch 是一个针对 Darwin Streaming Server 的软件包，它包含了源代码、补丁文件以及相关的更新内容。这个组合主要用于升级或修复已安装的 ...
DataSphere Studio 1.1.0的安装
qq_44665283的博客
11-16 4219
DataSphere Studio一键安装文档 一、使用前环境准备 a. 基础软件安装 Linkix需要的命令工具（在正式安装前，脚本会自动检测这些命令是否可用，如果不存在会尝试自动安装，安装失败则需用户手动安装以下基础shell命令工具）： telnet tar sed dos2unix mysql yum java unzip expect yum install -y telnet tar sed dos2unix mysql unzip expect 需要安
开源里程碑 | 数据应用开发管理集成框架 DataSphere Studio 1.1.0 新版本发布
weixin_39709847的博客
07-05 794
DataSphere Studio 简介DataSphere Studio（简称 DSS）是微众银行自研的数据应用开发管理集成框架。基于插拔式的集成框架设计，及计算中间件 Linkis ，可轻松接入上层各种数据应用系统，让数据开发变得简洁又易用。在统一的 UI 下，DataSphere Studio 以工作流式的图形化拖拽开发体验，将满足从数据交换、脱敏清洗、分析挖掘、质量检测、可视化展现、定时调度到数据输出应用等，数据应用开发全流程场景需求。DSS 通过插拔式的集成框架设计，让用户可以根据需要，简单快速替
探索DataSphereStudio：一体化数据开发与治理平台
最新发布
gitblog_00059的博客
04-25 406
探索DataSphereStudio：一体化数据开发与治理平台 项目地址:https://gitcode.com/WeBankFinTech/DataSphereStudio 项目简介 DataSphereStudio 是由微众银行开源的一款全方位数据研发和治理工具。它旨在提供一个统一的工作台，让数据科学家、数据工程师、BI分析师等角色可以协作完成数据处理、模型训练、数据分析及可视化等一系列任务。...
dss安装部署文档（持续更新）
北凉徐凤年的博客
03-17 2215
dss安装部署文档
【运维】linkis安装dss保姆级教程与踩坑实践
hiliang521的博客
08-15 876
linkis 安装dss保姆级教程与踩坑实践
【学习版】WeDataSphere组件容器化部署&实战
qq_34226628的博客
05-21 2249
WeDataSphere
dss-plugin-nlp-google-cloud-translation:Dataiku DSS插件可使用基本的Google Cloud转换API
03-12
为了增强其功能，DSS支持插件系统，其中就包括了对自然语言处理（NLP）的支持，例如“dss-plugin-nlp-google-cloud-translation”。 **Google Cloud Translation API** Google Cloud Translation API是谷歌提供的...
DH 软件平台 DH-DSS-H8900S2-B 智慧园区综合管理平台-智慧园区APP业务介绍和配置说明 V3.001.0000
11-06
DH-DSS-H8900S2-B智慧园区综合管理平台是专为现代园区打造的一款高效、智能的管理系统，旨在提升园区的运营管理效率和服务质量。该平台的核心在于其智慧园区APP，它集成了多种功能，涵盖了园区生活的各个方面，旨在...
DSS-Steam
03-30
1. **环境搭建**：安装Node.js和Vue CLI，创建一个新的Vue项目，然后将DSS-Steam的代码导入。 2. **组件设计**：根据需求设计并实现各个功能组件，如游戏列表组件、搜索组件、用户登录/注册组件等。 3. **路由配置...
Linkis1.0.2 安装及使用指南
zyz_Alvin的博客
10-18 4554
Linkis 安装及使用指南 1. 背景 2. 简介 2.1 Linkis 2.1.1 核心特性 2.2 DataSphereStudio 3. 安装 3.1 涉及组件版本说明 3.2 依赖环境安装 3.3 安装包准备 3.4 安装 3.4.1 安装环境检查 3.4.1.1 硬件环境检查 3.4.1.2 依赖环境检查 3.4.1.3 安装用户检查 3.4.1.4 安装命令检查 3.4.1.5 目录检查 3.4.2 解压安装包 3.4.3 修改配置 3.4.4 安装目录与配置检查...
【DataSphereStudio - 1.1.0 安装介绍】
haoheiao的博客
11-08 2093
DataSphere Studio（简称 DSS）是微众银行自研的数据应用开发管理集成框架。基于插拔式的集成框架设计，及计算中间件 Linkis ，可轻松接入上层各种数据应用系统，让数据开发变得简洁又易用。在统一的 UI 下，DataSphere Studio 以工作流式的图形化拖拽开发体验，将满足从数据交换、脱敏清洗、分析挖掘、质量检测、可视化展现、定时调度到数据输出应用等，数据应用开发全流程场景需求。【DataSphereStudio - 1.1.0 安装介绍】
DataSphereStudio1.1.2升级linkis到1.5.0
张顶飞的博客
01-29 257
linkis从1.4.0升级到1.5.0之后，部分子包的artifactId发生了变化，导致DataSphereStudio1.1.2无法直接从linkis-1.4.0升级到linkis-1.5.0。
dss_linkis(datasphere studio-1.1.1、linkis-1.1.1)基础框架安装
weixin_44382736的博客
01-27 1363
dss_linkis(datasphere studio-1.1.1、linkis-1.1.1)基础框架安装
开源大数据开发平台DataSphereStudio&Linkis踩坑记录
热门推荐
ck978105293的博客
06-09 1万+
DataSphere Studio（DSS）是微众银行开源的一站式大数据开发平台，开源于19年7月左右，目前市面上只发现这一个产品，，基于公司需求，我们调研后发现基本满足现有的需求，于是安装，安装过程踩了些坑，记录分享下。 环境： centOS7 DSS 0.7.0 Linkis 0.9.3 1.spark任务请求资源报错，版本不兼容 微众使用的hadoop版本是2.7.2，spark使用的是2.4.3，spark内hadoop的相关的包是2.7.3版本，在安装成功后请求spar...
DataSphere Studio - 1.1.0安装部署 （单机版）
Xiaoweidumpb
08-01 909
DataSphere Studio - 1.1.0安装部署 （单机版）
里程碑 | WeDataSphere 一站式开源大数据平台套件全面升级
张伟的专栏
05-15 689
国家 2035 远景规划提出要加快全面数字化转型的步伐，而“大数据平台”是数字化转型的基础技术之一。对于任何企业来说，建立和维护一个大数据平台都不是一件容易的事情，而建设一个有特色的、完整易用的大数据平台，显然更是一件技术难度极高的事情。基础平台由数据交换、数据分发、计算、存储四大层次组成，关注底层数据传输计算存储能力；功能平台由平台工具、数据工具、应用工具三大层次组成，关注用户各类功能工具需求实现。形成了完整的大数据平台技术体系，提供一站式的丰富数据平台组件及功能支撑。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

沧海之巅

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值