centos7.9上面搭建superset开发环境

月夜楓

已于 2022-07-06 16:12:24 修改

阅读量861

点赞数

分类专栏： linux 文章标签： python linux superset

于 2021-10-15 15:54:15 首次发布

本文链接：https://blog.csdn.net/cyxinda/article/details/120782046

版权

linux 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

centos7.9上面搭建superset开发环境

公司需要在superset上面接入sso单点登录，目前superset还不支持，需要在superset上面进行二次开发。曾经尝试过在windows10上面搭建开发环境，无奈python虚拟环境运行的各种失败后，放弃。
也尝试了在Ubuntu20.04上面搭建开发环境，最终npm打包功能不断失败，最终也放弃，在Ubuntu16上面尝试安装能够成功，但是桌面UI做的太烂了，遂放弃。
总结了下，Ubuntu20.04的桌面做的很炫，比windows做的漂亮，但是内核版本太高5.11，对于目前很多软件的兼容性不是太好。windows就别提了，superset官网说的好像是放弃了windows环境上面的开发。所以，最后尝试在centos7.9的桌面版本上面部署开发环境得以成功，记录本文如下。

在win10上面centos7.9桌面版本虚拟机

安装VMware

在VMware上面创建centos虚拟机

参考文档就不写了，网上搜索一大堆；
开通防火墙
firewall-cmd --zone=public --add-port=9000/tcp --permanent
firewall-cmd --zone=public --add-port=8088/tcp --permanent
挂载磁盘过程，参考：https://www.cnblogs.com/yaun1498078591/p/11970518.html

装anaconda环境

下载anaconda
进入到anaconda官网下载linux版本：https://www.anaconda.com/download/#linux

安装anaconda

#按照漫长的提示，完成安装anaconda
[cyxinda@centos7 superset]$ bash Anaconda3-2021.05-Linux-x86_64.sh
# 加载环境变量
[cyxinda@centos7 superset]$ source ~/.bashrc	
[cyxinda@centos7 superset]$ conda --version
conda 4.10.1

创建虚拟环境

#创建虚拟环境
(base) [cyxinda@centos7 superset]$ conda create -n python3.8.11 python=3.8.11

(python3.8.11) [cyxinda@centos7 superset]$ conda env list
# conda environments:
#
base                     /home/cyxinda/anaconda3
python3.8.11          *  /home/cyxinda/anaconda3/envs/python3.8.11

#切换虚拟环境
(base) [cyxinda@centos7 superset]$ source activate python3.8.11
(python3.8.11) [cyxinda@centos7 superset]$

修改pip源

(python3.8.11) [cyxinda@centos7 ~]$ mkdir ~/.pip
(python3.8.11) [cyxinda@centos7 ~]$ vim ~/.pip/pip.conf
[global]
trusted-host =  mirrors.aliyun.com
index-url = http://mirrors.aliyun.com/pypi/simple/

下载superset源码

加速github

如下两种方法任意即可

方法一：
加速教程，参考：https://zhuanlan.zhihu.com/p/75994966?utm_source=wechat_session
在https://www.ipaddress.com/网站上，获取host和ip对应的地址
编辑/etc/hosts
bash 140.82.112.4 github.com 199.232.69.194 github.global.ssl.fastly.net 104.16.16.35 registry.npmjs.org
方法二：
替换源地址：
https://github.com/xxx.git
替换为：
https://github.com.cnpmjs.org/xxx.git
示例：
git clone https://github.com.cnpmjs.org/xxx.git
说白了，就在http://github.com后面加个http://cnpmjs.org

下载superset源码

superset的github托管地址为：https://github.com/apache/superset
找到superset的1.3.1release下载地址：https://github.com/apache/superset/releases/tag/1.3.1
下载源码：

(python3.8.11) [cyxinda@centos7 superset]$ wget https://github.com/apache/superset/archive/refs/tags/1.3.1.zip
(python3.8.11) [cyxinda@centos7 superset]$ unzip 1.3.1.zip
(python3.8.11) [cyxinda@centos7 superset]$ ls
1.3.1.zip  Anaconda3-2021.05-Linux-x86_64.sh  install.sh  superset-1.3.1

目录结构如下：

code
└── superset
    ├── 1.3.1.zip
    ├── Anaconda3-2021.05-Linux-x86_64.sh
    ├── install.sh
    └── superset-1.3.1
        ├── CHANGELOG.md
        ├── CODE_OF_CONDUCT.md
        ├── CONTRIBUTING.md
        ├── docker
        ├── docker-compose-non-dev.yml
        ├── docker-compose.yml
        ├── Dockerfile
        ├── docs
        ├── helm
        ├── INSTALL.md
        ├── LICENSE.txt
        ├── lintconf.yaml
        ├── Makefile
        ├── MANIFEST.in
        ├── NOTICE
        ├── pytest.ini
        ├── README.md
        ├── RELEASING
        ├── requirements
        ├── RESOURCES
        ├── scripts
        ├── setup.cfg
        ├── setup.py
        ├── superset
        ├── superset-frontend
        ├── superset-websocket
        ├── tests
        ├── tox.ini
        └── UPDATING.md

构建superset

根据开发文档，构建superset
文档地址：https://github.com.cnpmjs.org/apache/superset/blob/master/CONTRIBUTING.md

安装依赖

(python3.8.11) [cyxinda@centos7 superset-frontend]$ pwd
/code/superset/superset-1.3.1/
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ pip install -r requirements/testing.txt
Looking in indexes: http://mirrors.aliyun.com/pypi/simple/
Obtaining file:///. (from -r requirements/base.txt (line 8))
Collecting aiohttp==3.7.2
  Downloading http://mirrors.aliyun.com/pypi/packages/63/c8/7c58c9ef4b7ac8f12c83b21e20e6566c48660ae49e403afa025d01df2550/aiohttp-3.7.2-cp38-cp38-manylinux2014_x86_64.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 1.8 MB/s
# 经过漫长的等待，以及调错即可完成

安装superset

(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ pwd
/code/superset/superset-1.3.1
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ pip install -e .

创建管理员用户

# Create an admin user in your metadata database (use `admin` as username to be able to load the examples)
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ superset fab create-admin
Username [admin]: admin
User first name [admin]: cao
User last name [user]: yong
Email [admin@fab.org]: caoyong1@lenovo.com
Password:
Repeat for confirmation:
Recognized Database Authentications.
Admin User admin created.
# Initialize the database
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ superset db upgrade
# Create default roles and permissions
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ superset init

异常情况1：
ERROR: No matching distribution found for mysqlclient==1.4.2.post1
参考：https://blog.csdn.net/w710537643/article/details/119573849
需要安装
```
sudo yum install python3-devel mysql-devel 
```
异常情况2：
Running setup.py install for sasl … error
参考：https://blog.csdn.net/vkingnew/article/details/86506876
需要安装依赖包：
```
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ sudo yum -y install cyrus-sasl cyrus-sasl-devel cyrus-sasl-lib
```

构建前端资源

下载nvm安装文件

(python3.8.11) [cyxinda@centos7 superset-frontend]$ get https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.0/install.sh
#执行nvm安装
(python3.8.11) [cyxinda@centos7 superset]$ bash install.sh
=> Downloading nvm from git to '/home/cyxinda/.nvm'
=> Cloning into '/home/cyxinda/.nvm'...
remote: Enumerating objects: 317, done.
remote: Counting objects: 100% (317/317), done.
remote: Compressing objects: 100% (269/269), done.
remote: Total 317 (delta 36), reused 136 (delta 23), pack-reused 0
Receiving objects: 100% (317/317), 170.27 KiB | 0 bytes/s, done.
Resolving deltas: 100% (36/36), done.
=> Compressing and cleaning up git repository

=> Appending nvm source string to /home/cyxinda/.bashrc
=> Appending bash_completion source string to /home/cyxinda/.bashrc
=> Close and reopen your terminal to start using nvm or run the following to use                                                                   it now:

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads                                                                   nvm bash_completion
(python3.8.11) [cyxinda@centos7 superset]$ nvm
bash: nvm: command not found...
(python3.8.11) [cyxinda@centos7 superset]$ source ~/.bashrc
(base) [cyxinda@centos7 superset]$ nvm --version
0.37.0
(base) [cyxinda@centos7 superset-frontend]$ nvm use --lts
Now using node v14.18.1 (npm v6.14.15)

安装npm和nodejs

(base) [cyxinda@centos7 superset-frontend]$ pwd
/code/superset/superset-1.3.1/superset-frontend
(base) [cyxinda@centos7 superset-frontend]$ nvm install --lts
Installing latest LTS version.
Downloading and installing node v14.18.1...
Downloading https://nodejs.org/dist/v14.18.1/node-v14.18.1-linux-x64.tar.xz...
########################################################################################################################################### 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v14.18.1 (npm v6.14.15)
Creating default alias: default -> lts/* (-> v14.18.1)
(base) [cyxinda@centos7 superset-frontend]$ npm -version
6.14.15

安装node依赖包

(base) [cyxinda@centos7 superset-frontend]$ pwd
/code/superset/superset-1.3.1/superset-frontend
(base) [cyxinda@centos7 superset-frontend]$ npm ci
#漫长的等待过程，可能重复执行几次才能成功。。。。

安装前端资源

(base) [cyxinda@centos7 superset-frontend]$ npm run build
#经过漫长的等待

搭建superset开发环境

下载pycharm IDE
首先在下面的网址下载安装包：
https://www.jetbrains.com/pycharm/download/#section=linux

(python3.8.11) [cyxinda@centos7 pycharm]$ pwd
/code/ide/pycharm
(python3.8.11) [cyxinda@centos7 pycharm]$ tar -xvf pycharm-community-2021.2.2.tar.gz
(python3.8.11) [cyxinda@centos7 pycharm]$ cd pycharm-community-2021.2.2/
(python3.8.11) [cyxinda@centos7 pycharm-community-2021.2.2]$ pwd
/code/ide/pycharm/pycharm-community-2021.2.2
(python3.8.11) [cyxinda@centos7 pycharm-community-2021.2.2]$ cd bin

在centos桌面上面启动pycharm
可以直接使用脚本启动：

(python3.8.11) [cyxinda@centos7 bin]$ ./pycharm.sh

并且，为了方便起见，可以创建桌面的快捷方式

[root@centos7 ~]# vim /usr/share/applications/Pycharm.desktop
#在Pycharm.desktop中编辑如下内容，强制保存：
[Desktop Entry]
Type=Application
Name=Pycharm
GenericName=Pycharm3
Comment=Pycharm3:The Python IDE
Exec=sh /code/ide/pycharm/pycharm-community-2021.2.2/bin/pycharm.sh
Icon=/code/ide/pycharm/pycharm-community-2021.2.2/bin/pycharm.png
Terminal=pycharm
Categories=Pycharm;

在桌面上面出现快捷方式如下：
在这里插入图片描述
单击快捷方式，打开pycharm：

启动superset
打开位于/code/superset/superset-1.3.1的项目：点击Open按钮图标，选择位置
设置Python Interpreter
在Settings中，设置上文中创建好的python Interpreter，如下图：

创建启动程序
在目录/code/superset/superset-1.3.1/中创建superset_run.py
在这里插入图片描述
内容如下：

from superset.app import create_app
app = create_app()
if __name__ == '__main__':
    app.debug = True # 设置调试模式，生产模式的时候要关掉debug
    app.run(host='0.0.0.0', port=8888, debug=True)

debug模式启动superset_run.py

/home/cyxinda/anaconda3/envs/python3.8.11/bin/python /code/ide/pycharm/pycharm-community-2021.2.2/plugins/python-ce/helpers/pydev/pydevd.py --multiproc --qt-support=auto --client 127.0.0.1 --port 46778 --file /code/superset/superset-1.3.1/superset_run.py
Connected to pydev debugger (build 212.5284.44)
logging was configured successfully
2021-10-16 03:24:59,197:INFO:superset.utils.logging_configurator:logging was configured successfully
2021-10-16 03:24:59,220:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/home/cyxinda/anaconda3/envs/python3.8.11/lib/python3.8/site-packages/flask_caching/__init__.py:201: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
  warnings.warn(
 * Serving Flask app "superset.app" (lazy loading)
 * Environment: development
 * Debug mode: on
2021-10-16 03:25:08,026:INFO:werkzeug: * Running on http://0.0.0.0:8888/ (Press CTRL+C to quit)
2021-10-16 03:25:08,029:INFO:werkzeug: * Restarting with stat
2021-10-16 03:25:13,091:INFO:superset.utils.logging_configurator:logging was configured successfully
2021-10-16 03:25:13,113:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/home/cyxinda/anaconda3/envs/python3.8.11/lib/python3.8/site-packages/flask_caching/__init__.py:201: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
  warnings.warn(
2021-10-16 03:25:21,591:WARNING:werkzeug: * Debugger is active!
2021-10-16 03:25:21,650:INFO:werkzeug: * Debugger PIN: 320-957-409

前端访问验证

在/code/superset/superset-1.3.1/superset/initialization/init.py的723行，添加断点
输入用户名密码admin/****后：

后端：
F9继续执行
前端

至此，superset在centos7.9上面的开发环境，搭建完毕。

如上，是通过安装过程启动superset，但是实际开发，应该是前后端分离的。所以，应该先启动后端项目；再启动前端项目，分开启动，分开调试。

后端在pycharm中直接启动，不用执行npm run build，启动了默认端口是8088
启动vscode，加载前端项目/code/superset/superset-1.3.1/superset-frontend，如下图：

如果后端项目启动的不是默认的8088端口，而是8081，则在vscode中，需要执行的命令变为：npm run dev-server – --supersetPort=8081

接下来就可以使用vscode进行代码调试了。

月夜楓

关注

0
点赞
踩
7

收藏

觉得还不错? 一键收藏
2
评论
centos7.9上面搭建superset开发环境

装anaconda环境下载anaconda进入到anaconda官网下载linux版本：https://www.anaconda.com/download/#linux安装anaconda#按照漫长的提示，完成安装anaconda[cyxinda@centos7 superset]$ bash Anaconda3-2021.05-Linux-x86_64.sh# 加载环境变量[cyxinda@centos7 superset]$ source ~/.bashrc [cyxinda@cent.
复制链接

扫一扫