python搭建

新人入职须知–（2）

dmc各模块及简单流程介绍

dmc模块及简单流程介绍
除了上述模块，dmc核心模块正在使用java重构为hora模块。目前hora已部分取代thanos的功能。

---

python模块配置方法

（适用于fe、thanos、pandora、tassadar、overlord、behemoth、bdp-di）

通用配置：

1. git克隆代码到本地（注意项目包含子模块），pycharm打开项目工程。

2. 配置远程传输路径。
   1）.点击Tools–Deployment–Configuration。点击左上角+号，命名Deployment，点击ok。

   • Type：SFTP
   • SSH configuration：配置dev121的连接信息。不勾选Visible only for this project，则一次配置后可以在其他配置时复用。
   • Root path：配置服务器上项目存放的根目录。
   • Web server URL：不需配置
   • 切换到Mappings标签页，配置Deplyment path。注意配置的Root Path和Deployment Path构成项目在服务器上的完整存放路径。例如Root Path配置为/home/fudanchen, Deployment Path配置为/bdp-fe，项目传输到服务器上的实际路径为/home/fudanchen/bdp-fe。

   2）.在connection标签页勾选Visible only for this project，避免多个项目配置过多造成混乱。
   3）.点击ok，配置完成。

3. 代码传输到远程服务器。
   在项目根目录下点击右键–Deployment–Upload to，选择刚刚配置的传输通道，开始进行文件传输。点击Tools–Deployment–Browse Remote Host可以看到远程服务器路径。

4. 配置远程解释器

   • Pycharm–preference，选择Project–Project Interpreter–Add。
   • 选择SSH Interpreter–Existing server configuration，选择之前配置的的SSH通道，点击next–finish。
   • Project Interpreter选择刚刚配置的SSH远程解释器，Path mappings配置本地代码工程和服务器代码工程的对应路径。配置完成后点击ok。

5. 配置文件
   在项目conf目录下新建cuntom.yaml文件，配置其他服务和数据库地址。
   配置文件截图

模块配置

6. 启动脚本配置
   fe的启动脚本是src/server/mcserver.py
   run–Edit Configurations

   • Script Path：选择mcserver.py
   • Parameters：–port=自定义端口号 --environment=offline
   • Environment variablesPYTHONUNBUFFERED=1;FE_HOME=/home/fudanchen/dmc-core/bdp-fe;CONF=/home/fudanchen/dmc-core/bdp-fe/conf;BIN=/home/fudanchen/dmc-core/bdp-fe/bin;PID=/app/logs/bdp-fe;LIB=/home/fudanchen/dmc-core/bdp-fe/lib;SRC=/home/fudanchen/dmc-core/bdp-fe/src;DATA=/home/fudanchen/dmc-core/bdp-fe/data;TEMPLATES=/home/fudanchen/dmc-core/bdp-fe/templates;SERVER=/home/fudanchen/dmc-core/bdp-fe/src/server;LOCALE=/home/fudanchen/dmc-core/bdp-fe/locale;TASK=/home/fudanchen/dmc-core/bdp-fe/src/task;SCRIPT=/home/fudanchen/dmc-core/bdp-fe/src/script;TEST_RESOURCE=/home/fudanchen/dmc-core/bdp-fe/test/resource;PYTHON=python;DATE=`date +"%Y-%m-%d"`;PYTHONPATH=/home/fudanchen/dmc-core/bdp-fe/lib:/home/fudanchen/dmc-core/bdp-fe/lib/pyhs2:/home/fudanchen/dmc-core/bdp-fe/lib/push:/home/fudanchen/dmc-core/bdp-fe/lib/dateutil:/home/fudanchen/dmc-core/bdp-fe/src:/home/fudanchen/dmc-core/bdp-fe/src/common/cloghandler

   （其中的/home/fudanchen/dmc-core换成自己的远程代码路径)

7. 运行，出现start mc api server at 端口号，即为启动成功，可以通过dev121.haizhi.com:端口号访问服务。

其他服务的配置与fe类似，需要修改启动脚本和启动配置。

• thanos
  • 启动脚本：server.py
  • Parameters：–port=端口号
  • Environment variables：PYTHONUNBUFFERED=1;LIB=/home/fudanchen/dmc-core/thanos/lib;PYTHONPATH=/home/fudanchen/dmc-core/thanos/src:/home/fudanchen/dmc-core/thanos/lib;CONF=/home/fudanchen/dmc-core/thanos/conf;LOCALE=/home/fudanchen/dmc-core/thanos/locale;LOGS=/home/fudanchen/dmc-core/thanos
• pandora
  • 启动脚本：src/server/server.py
  • Parameters：–port=端口号
  • Environment variables：PYTHONUNBUFFERED=1;PANDORA_HOME=/home/fudanchen/dmc-core/pandora;LOGS=/home/fudanchen/dmc-core/pandora/logs;CONF=/home/fudanchen/dmc-core/pandora/conf;BIN=/home/fudanchen/dmc-core/pandora/bin;PID=/home/fudanchen/dmc-core/pandora/logs;LIB=/home/fudanchen/dmc-core/pandora/lib;SRC=/home/fudanchen/dmc-core/pandora/src;DATA=/home/fudanchen/dmc-core/pandora/data;TEMPLATES=/home/fudanchen/dmc-core/pandora/templates;SERVER=/home/fudanchen/dmc-core/pandora/src/server;LOCALE=/home/fudanchen/dmc-core/pandora/locale;TASK=/home/fudanchen/dmc-core/pandora/src/task;SCRIPT=/home/fudanchen/dmc-core/pandora/src/script;TEST_RESOURCE=/home/fudanchen/dmc-core/pandora/test/resource;PYTHONPATH=/home/fudanchen/dmc-core/pandora/lib:/home/fudanchen/dmc-core/pandora/lib/pyhs2:/home/fudanchen/dmc-core/pandora/lib/push:/home/fudanchen/dmc-core/pandora/lib/dateutil:/home/fudanchen/dmc-core/pandora/src:/home/fudanchen/dmc-core/pandora/src/common/cloghandler;PYTHON=python;DATE=date +"%Y-%m-%d"
• tassadar
  • 启动脚本：server.py
  • Parameters：–port=端口号
  • Environment variables：PYTHONUNBUFFERED=1;TASSADAR_HOME=/home/fudanchen/dmc-core/tassadar;PYTHONPATH=/home/fudanchen/dmc-core/tassadar/src;CONF=/home/fudanchen/dmc-core/tassadar/conf
• overlord
  • 启动脚本：src/server/overlord.py
  • Parameters：无
  • Environment variables：PYTHONUNBUFFERED=1;OVERLORD_HOME=/home/fudanchen/dmc-core/overlord;LOGS=/home/fudanchen/dmc-core/overlord/logs;CONF=/home/fudanchen/dmc-core/overlord/conf;BIN=/home/fudanchen/dmc-core/overlord/bin;PID=/home/fudanchen/dmc-core/overlord/pid;LIB=/home/fudanchen/dmc-core/overlord/lib;SRC=/home/fudanchen/dmc-core/overlord/src;DATA=/home/fudanchen/dmc-core/overlord/data;SERVER=/home/fudanchen/dmc-core/overlord/src/server;TEST=/home/fudanchen/dmc-core/overlord/test;PYTHONPATH=/home/fudanchen/dmc-core/overlord/lib:/home/fudanchen/dmc-core/overlord/lib/cloghandler:/home/fudanchen/dmc-core/overlord/src
• behemoth
  • 启动脚本：src/server/behemoth.py
  • Parameters：–port=端口号 --environment=offline
  • Environment variables：PYTHONUNBUFFERED=1;BEHEMOTH_HOME=/home/fudanchen/dmc-core;LOGS=/home/fudanchen/dmc-core/behemoth/logs;CONF=/home/fudanchen/dmc-core/behemoth/conf;BIN=/home/fudanchen/dmc-core/behemoth/bin;LIB=/home/fudanchen/dmc-core/behemoth/lib;SRC=/home/fudanchen/dmc-core/behemoth/src;SCRIPT=/home/fudanchen/dmc-core/behemoth/script;SERVER=/home/fudanchen/dmc-core/behemoth/src/server;PYTHONPATH=/home/fudanchen/dmc-core/behemoth/lib:/home/fudanchen/dmc-core/behemoth/lib/cloghandler:/home/fudanchen/dmc-core/behemoth/src:/home/fudanchen/dmc-core/behemoth/src/util:/home/fudanchen/dmc-core/behemoth/script;PYTHON=$(which python);DATE=date +"%Y-%m-%d";MODULE_NAME=behemoth
• bdp-di
  • 启动脚本：src/server/di_server.py
  • Parameters：–environment=offline
  • Environment variables：PYTHONUNBUFFERED=1;PYTHONPATH=/home/fudanchen/dmc-core/bdp-di/src;CONF=/home/fudanchen/dmc-core/bdp-di/conf;SRC=/home/fudanchen/dmc-core/bdp-di/src