尚硅谷大数据superset安装包冲突

尚硅谷大数据superset安装包冲突

一、安装Miniconda

1. 下载Miniconda（Python3版本）

   下载地址：https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

2. 执行以下命令进行安装，并按照提示操作，直到安装完成。

   bash Miniconda3-latest-Linux-x86_64.sh
   

3. 在安装过程中，出现以下提示时，可以指定安装路径

4. 加载环境变量配置文件，使之生效

   source ~/.bashrc
   

5. 取消激活base环境

   conda config --set auto_activate_base false
   

二、创建Python3.7环境

1. 配置conda国内镜像

   conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
   conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
   conda config --set show_channel_urls yes
   

2. 创建Python3.9环境

   conda create --name superset python=3.9
   

3. 激活superset环境

   conda activate superset
   

三、Superset部署

1. 安装Superset之前，需安装以下所需依赖

   sudo yum install -y gcc gcc-c++ libffi-devel python-devel python-pip python-wheel python-setuptools openssl-devel cyrus-sasl-devel openldap-devel
   

2. 安装superset

   • 安装（更新）setuptools和pip

     pip install --upgrade setuptools pip -i https://pypi.douban.com/simple/
     

   • 安装superset

     pip install apache-superset -i https://pypi.douban.com/simple/
     

   • 初始化supetset数据库superset db upgrade出现如下错误

     Failed to create app
     Traceback (most recent call last):
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/app.py", line 37, in create_app
         app_initializer.init_app()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 460, in init_app
         self.init_app_in_ctx()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 409, in init_app_in_ctx
         self.configure_url_map_converters()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 508, in configure_url_map_converters
         from superset.utils.url_map_converters import (
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/utils/url_map_converters.py", line 21, in <module>
         from superset.models.tags import ObjectTypes
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/models/__init__.py", line 17, in <module>
         from . import core, datasource_access_request, dynamic_plugins, sql_lab, user_attributes
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/models/core.py", line 63, in <module>
         from superset.utils import cache as cache_util, core as utils
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/utils/cache.py", line 28, in <module>
         from werkzeug.wrappers.etag import ETagResponseMixin
     ModuleNotFoundError: No module named 'werkzeug.wrappers.etag'
     

     解决办法：一定要先统一Flask和Werkzeug框架的版本，不然问题还是无法解决

     pip uninstall Flask
     pip uninstall Werkzeug
     
     pip install Flask==2.0.2
     pip install Werkzeug==2.0.2
     

     再次进行初始化出现superset db upgrade

     出现如下错误：

     --------------------------------------------------------------------------------
                                         WARNING
     --------------------------------------------------------------------------------
     A Default SECRET_KEY was detected, please use superset_config.py to override it.
     Use a strong complex alphanumeric string and use a tool to help you generate 
     a sufficiently random sequence, ex: openssl rand -base64 42
     --------------------------------------------------------------------------------
     --------------------------------------------------------------------------------
     logging was configured successfully
     2022-09-30 15:19:27,622:INFO:superset.utils.logging_configurator:logging was configured successfully
     2022-09-30 15:19:27,644:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
     Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
     2022-09-30 15:19:27,651:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
     Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
     2022-09-30 15:19:27,658:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
     No PIL installation found
     2022-09-30 15:19:27,987:INFO:superset.utils.screenshots:No PIL installation found
     Failed to create app
     Traceback (most recent call last):
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/app.py", line 37, in create_app
         app_initializer.init_app()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 460, in init_app
         self.init_app_in_ctx()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 410, in init_app_in_ctx
         self.configure_data_sources()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 476, in configure_data_sources
         ConnectorRegistry.register_sources(module_datasource_map)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/connector_registry.py", line 42, in register_sources
         module_obj = __import__(module_name, fromlist=class_names)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/sqla/__init__.py", line 17, in <module>
         from . import models, views
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/sqla/views.py", line 28, in <module>
         from wtforms.ext.sqlalchemy.fields import QuerySelectField
     ModuleNotFoundError: No module named 'wtforms.ext'
     2022-09-30 15:19:28,607:ERROR:superset.app:Failed to create app
     Traceback (most recent call last):
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/app.py", line 37, in create_app
         app_initializer.init_app()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 460, in init_app
         self.init_app_in_ctx()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 410, in init_app_in_ctx
         self.configure_data_sources()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 476, in configure_data_sources
         ConnectorRegistry.register_sources(module_datasource_map)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/connector_registry.py", line 42, in register_sources
         module_obj = __import__(module_name, fromlist=class_names)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/sqla/__init__.py", line 17, in <module>
         from . import models, views
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/sqla/views.py", line 28, in <module>
         from wtforms.ext.sqlalchemy.fields import QuerySelectField
     ModuleNotFoundError: No module named 'wtforms.ext'
     Traceback (most recent call last):
       File "/home/hdfs/miniconda3/envs/superset/bin/superset", line 8, in <module>
         sys.exit(superset())
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
         return self.main(*args, **kwargs)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/flask/cli.py", line 600, in main
         return super().main(*args, **kwargs)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/click/core.py", line 1055, in main
         rv = self.invoke(ctx)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/click/core.py", line 1654, in invoke
         super().invoke(ctx)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
         return ctx.invoke(self.callback, **ctx.params)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/click/core.py", line 760, in invoke
         return __callback(*args, **kwargs)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
         return f(get_current_context(), *args, **kwargs)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/flask/cli.py", line 443, in decorator
         with __ctx.ensure_object(ScriptInfo).load_app().app_context():
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/flask/cli.py", line 406, in load_app
         app = locate_app(self, import_name, name)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/flask/cli.py", line 276, in locate_app
         return find_best_app(script_info, module)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/flask/cli.py", line 68, in find_best_app
         app = call_factory(script_info, app_factory)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/flask/cli.py", line 123, in call_factory
         return app_factory(*args, **kwargs)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/app.py", line 44, in create_app
         raise ex
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/app.py", line 37, in create_app
         app_initializer.init_app()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 460, in init_app
         self.init_app_in_ctx()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 410, in init_app_in_ctx
         self.configure_data_sources()
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/initialization/__init__.py", line 476, in configure_data_sources
         ConnectorRegistry.register_sources(module_datasource_map)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/connector_registry.py", line 42, in register_sources
         module_obj = __import__(module_name, fromlist=class_names)
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/sqla/__init__.py", line 17, in <module>
         from . import models, views
       File "/home/hdfs/miniconda3/envs/superset/lib/python3.9/site-packages/superset/connectors/sqla/views.py", line 28, in <module>
         from wtforms.ext.sqlalchemy.fields import QuerySelectField
     ModuleNotFoundError: No module named 'wtforms.ext'
     

     安装如下包即可解决

     pip install WTForms==2.3.3
     

3. 创建管理员用户

   export FLASK_APP=superset
   superset fab create-admin
   

4. superset初始化

   superset init
   

四、启动Supterset

1. 安装gunicorn

   pip install gunicorn -i https://pypi.douban.com/simple/
   

2. 启动

   gunicorn --workers 5 --timeout 120 --bind hadoop:8787  "superset.app:create_app()" --daemon 
   

3. 停止

   ps -ef | awk '/superset/ && !/awk/{print $2}' | xargs kill -9