2021SC@SDUSC
目录
简介
云存储是一种网上在线存储(英语:Cloud storage)的模式,即把数据存放在通常由第三方托管的多台虚拟服务器,而非专属的服务器上。山大智云是基于seafile开源框架开发的云存储平台。Seafile 是一款企业级文件托管平台,拥有高可靠性和高性能的特点。Seafile 包含一个 Web 界面,您可以用它来管理和分享您的文件。同时,它还提供了两种客户端,同步盘客户端和挂载盘客户端。另外,它还提供 IOS 和 Android 客户端。
经过团队的讨论,最终确定了分工,我负责SeafDav(提供webdav标准传输)模块,效果是将网盘资料库挂载到本地,有将近98%的代码是python,通过分析代码可以对自己的工程代码水平有很大的改善。
操作系统
Ubuntu 18.04 (.1~.6)
前置操作
安装docker
-
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
-
docker run -it -p 8000:8000 -p 8080:8080 -p 8082:8082 -p 3000:3000 -p 9200:9200 --name seafile-ce-env ubuntu:18.04 bash
-
安装中文语言
apt-get install language-pack-zh-hans locale-gen zh_CN.UTF-8 echo "export LC_ALL=zh_CN.UTF-8">>/root/.bashrc source /root/.bashrc
如果出现警告也可以不用管。
locate
如果得到以下结果则说明已安装中文:
LANG= LANGUAGE= LC_CTYPE="zh_CN.UTF-8" LC_NUMERIC="zh_CN.UTF-8" LC_TIME="zh_CN.UTF-8" LC_COLLATE="zh_CN.UTF-8" LC_MONETARY="zh_CN.UTF-8" LC_MESSAGES="zh_CN.UTF-8" LC_PAPER="zh_CN.UTF-8" LC_NAME="zh_CN.UTF-8" LC_ADDRESS="zh_CN.UTF-8" LC_TELEPHONE="zh_CN.UTF-8" LC_MEASUREMENT="zh_CN.UTF-8" LC_IDENTIFICATION="zh_CN.UTF-8" LC_ALL=zh_CN.UTF-8
-
安装前置组件
-
apt-get update && apt-get upgrade -y
-
apt-get install -y ssh libevent-dev libcurl4-openssl-dev libglib2.0-dev uuid-dev intltool libsqlite3-dev libmysqlclient-dev libarchive-dev libtool libjansson-dev valac libfuse-dev python-dateutil cmake re2c flex sqlite3 python-pip python-simplejson git libssl-dev libldap2-dev libonig-dev vim vim-scripts wget cmake gcc autoconf automake mysql-client librados-dev libxml2-dev curl sudo telnet netcat unzip netbase ca-certificates apt-transport-https build-essential libxslt1-dev libffi-dev libpcre3-dev libz-dev xz-utils nginx pkg-config poppler-utils libmemcached-dev openjdk-11-jdk aria2 sudo
-
curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - apt-get install -y nodejs
-
python3.6 -m pip install --upgrade pip rm /usr/bin/python rm /usr/bin/python3 ln -s /usr/bin/python3.6 /usr/bin/python ln -s /usr/bin/python3.6 /usr/bin/python3 rm /usr/bin/pip rm /usr/bin/pip3 ln -s /usr/local/bin/pip3.6 /usr/bin/pip ln -s /usr/local/bin/pip3.6 /usr/bin/pip3 pip install python-cas djangosaml2 cffi sqlalchemy pymysql pillow pycryptodome configparser pylibmc django-pylibmc elasticsearch==5.5.0 elasticsearch-dsl==5.4.0 Django==2.2.14 future captcha django-statici18n django-post_office==3.3.0 django-webpack_loader==0.7.0 gunicorn mysqlclient django-picklefield==2.1.1 openpyxl qrcode django-formtools django-simple-captcha djangorestframework==3.11.1 python-dateutil requests pillow pyjwt pycryptodome requests_oauthlib mock nose exam splinter pytest pytest-django json5 pyyaml jinja2
-
-
准备数据库
apt-get install -y mariadb-server service mysql start mysqladmin -u root password 123456
-
建表
create database ccnet charset utf8; create database seafile charset utf8; create database seahub charset utf8;
-
创建master用户
CREATE USER 'master'@'localhost' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON *.* TO 'master'@'localhost';
密码123456
-
-
下载项目源代码
cd ~/ mkdir -p ~/dev/source-code cd ~/dev/source-code git clone https://github.com/SDU-NetdiskTeam/libevhtp.git git clone https://github.com/SDU-NetdiskTeam/libsearpc.git git clone https://github.com/SDU-NetdiskTeam/seafile-server.git git clone https://github.com/SDU-NetdiskTeam/seahub.git git clone https://github.com/SDU-NetdiskTeam/seahub-extra.git seahub_extra git clone https://github.com/SDU-NetdiskTeam/pro-set.git pro git clone https://github.com/SDU-NetdiskTeam/seafdav.git cd libsearpc/ git checkout 8.0 cd ../seafile-server git checkout develop cd ../seahub git checkout develop
-
编译并安装项目
cd ../libevhtp cmake -DEVHTP_DISABLE_SSL=ON -DEVHTP_BUILD_SHARED=OFF . make make install ldconfig cd ../libsearpc ./autogen.sh ./configure make make install ldconfig cd ../seafile-server ./autogen.sh ./configure --disable-fuse make make install ldconfig
-
创建配置文件
mkdir ~/dev/conf cd ~/dev/conf cat > ccnet.conf <<EOF [Database] ENGINE = mysql HOST = localhost PORT = 3306 USER = master PASSWD = 123456 DB = ccnet CONNECTION_CHARSET = utf8 CREATE_TABLES = true [General] SERVICE_URL = http://127.0.0.1:8000 EOF cat > seahub_settings.py <<EOF DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'seahub', 'USER': 'master', 'PASSWORD': '123456', 'HOST': 'localhost', 'PORT': '3306', } } FILE_SERVER_ROOT = 'http://127.0.0.1:8082' EOF cat > seafevents.conf <<EOF [DATABASE] type = mysql host = localhost port = 3306 username = master password = 123456 name = seahub [AUDIT] enabled = false [INDEX FILES] external_es_server = true es_host = 127.0.0.1 es_port = 9200 enabled = true interval = 10m highlight = fvh ## If true, indexes the contents of office/pdf files while updating search index ## Note: If you change this option from "false" to "true", then you need to clear the search index and update the index again. See the FAQ for details. index_office_pdf = true [OFFICE CONVERTER] port = 6000 host = 127.0.0.1 enabled = false workers = 1 [SEAHUB EMAIL] enabled = false ## interval of sending Seahub email. Can be s(seconds), m(minutes), h(hours), d(days) interval = 30m # Enable statistics [STATISTICS] enabled=false # Enable Aria2 offline download [OFFLINE DOWNLOAD] enabled = true tempdir = /tmp/offline-download workers = 10 ## time limit of download. Can be s(seconds), m(minutes), h(hours), d(days) time-limit = 900s # Enable auto deletion [AUTO DELETION] enabled = true interval = 1d EOF cat > seafile.conf <<EOF [database] type = mysql host = localhost port = 3306 user = master password = 123456 db_name = seafile connection_charset = utf8 create_tables = true [virus_scan] scan_command = clamscan virus_code = 1 nonvirus_code = 0 scan_interval = 60 # scan_size_limit = (size limit for files to be scanned) # scan_skip_ext = (a comma (',') separated list of file extensions to be ignored) # threads = (number of concurrent threads for scan, one thread for one file, default to 4) EOF cat > seafdav.conf <<EOF [WEBDAV] # Default is false. Change it to true to enable SeafDAV server. enabled = true port = 8080 # If you deploy seafdav behind nginx/apache, you need to modify "share_name". share_name = / EOF mkdir ~/dev/seafile-data cd ~/dev/seafile-data ln -s ../conf/seafile.conf ./ mkdir ~/dev/logs
seafile-server
启动
service mysql start
seaf-server -c /root/dev/conf -d /root/dev/seafile-data -D all -f -l - &
seafile-server默认后台运行。
关闭
kill `ps -aux | grep seaf-server | sed -n 1p | awk '{print $2}'`
(部署过程中这一步是不需要的)
seahub
配置环境变量
cd ~/dev/source-code/seahub/
export PYTHONPATH=/usr/local/lib/python3.6/site-packages/:/root/dev/source-code/seahub/thirdpart:$PYTHONPATH
export PYTHONPATH=/root/dev/source-code:/root/dev/source-code/pro/python:$PYTHONPATH
export CCNET_CONF_DIR=/root/dev/conf
export SEAFILE_CONF_DIR=/root/dev/seafile-data
export SEAFILE_CENTRAL_CONF_DIR=/root/dev/conf
export SEAFES_DIR=/root/dev/source-code/pro/python/seafes
export SEAHUB_DIR=/root/dev/source-code/seahub
export SEAHUB_LOG_DIR=/root/dev/logs
python manage.py migrate cd ../*_extra mysql
-
然后在mysql中输入以下内容:
-
use seahub; source raw_table_sql.sql
-
python manage.py createsuperuser
然后按提示输入邮箱(可以不真实存在)、密码。
-
启动,如果没创建
seahub
窗口,则创建seahub
窗口: -
screen -S seahub cd /root/dev/source-code/seahub python manage.py runserver 0.0.0.0:8000
最后回到主窗口。此时seahub在seahub窗口内并发运行。
(推荐在最后完成此步骤)
seahub-frontend
npm编译
cd ~/dev/souece-code/seahub/frontend
npm i core-js@2.6.9 -D
npm install
npm audit fix # 这步可省略
npm run-script build
cd ..
make clean
make dist
-
配置
vim /root/dev/conf/seahub_settings.py import os PROJECT_ROOT = '/root/dev/source-code/seahub' WEBPACK_LOADER = { 'DEFAULT': { 'BUNDLE_DIR_NAME': 'frontend/', 'STATS_FILE': os.path.join(PROJECT_ROOT, 'frontend/webpack-stats.dev.json'), } } DEBUG = True
-
启动
screen -S frontend cd /root/dev/source-code/seahub/frontend npm run dev
最后回到主窗口。此时seahub前端在frontend窗口内并发运行。
(推荐在最后完成此步骤)
seafevents
配置
cp /root/dev/source-code/pro/python/seafevents/main.py /root/dev/source-code/pro/python/seafevents/seafile_events.py
cat > /root/dev/source-code/pro/python/seafevents/run.sh << EOD
#!/bin/bash
# Your ccnet conf dir
export CCNET_CONF_DIR=/root/dev/conf
# Your seafile conf dir
export SEAFILE_CONF_DIR=/root/dev/seafile-data
# Your extra python path.
export PYTHONPATH=/usr/local/lib/python3.6/site-packages/:/root/dev/source-code/seahub/thirdpart:/root/dev/source-code:/root/dev/source-code/pro/python:/usr/local/lib/python3.6/site-packages/:/root/dev/source-code/seahub/thirdpart:
export PYTHONPATH=/root/dev/source-code:/root/dev/source-code/pro/python:/root/dev/source-code:/root/dev/source-code/pro/python:/usr/local/lib/python3.6/site-packages/:/root/dev/source-code/seahub/thirdpart:
pkill -f "python seafile_events.py"
python seafile_events.py --loglevel=info --logfile=/root/dev/logs/events.log --reconnect --config-file=/root/dev/conf/seafevents.conf
EOD
chmod +x /root/dev/source-code/pro/python/seafevents/run.sh
-
启动
-
screen -S seafevents cd /root/dev/source-code/pro/python/seafevents ./run.sh
最后回到主窗口。此时周期事件脚本在seafevents窗口内并发运行。
(推荐在最后完成此步骤)
Elastic Search
创建es用户:
groupadd es
useradd es -g es
passwd es
然后输入密码。接下来从proset中复制elastic search:
mkdir /home/es
cp -r /root/dev/source-code/pro/elasticsearch /home/es/ chown -R es /home/es
-
配置
cat > /root/dev/source-code/pro/python/seafes/run.sh <<EOF export PYTHONPATH=/usr/local/lib/python3.6/site-packages/:/root/dev/source-code/seahub/thirdpart:$PYTHONPATH export PYTHONPATH=/root/dev/source-code:/root/dev/source-code/pro/python:$PYTHONPATH export SEAHUB_DIR=/root/dev/source-code/seahub export CCNET_CONF_DIR=/root/dev/conf export SEAFILE_CONF_DIR=/root/dev/seafile-data export EVENTS_CONFIG_FILE=/root/dev/conf/seafevents.conf if [[ $# == 1 && $1 == "clear" ]]; then python -m seafes.index_local --loglevel debug clear else python -m seafes.index_local --loglevel debug update fi EOF chmod +x /root/dev/source-code/pro/python/seafes/run.sh
-
启动
screen -S seafes
然后在此窗口内:
最后回到主窗口。此时Elastic Search在es窗口内并发运行。
(推荐在最后完成此步骤)
-
切换至此用户:
su es
-
运行Elastic Search
cd /home/es/elasticsearch/bin ./elasticsearch
-
seafdav
配置
vim /root/dev/source-code/seafdav/run.sh
#!/bin/bash
export CCNET_CONF_DIR=/root/dev/conf
export SEAFILE_CONF_DIR=/root/dev/seafile-data
export SEAFILE_CENTRAL_CONF_DIR=/root/dev/conf
export PYTHONPATH=/usr/local/lib/python3.6/site-packages/:/root/dev/source-code/seahub/thirdpart:$PYTHONPATH
export PYTHONPATH=/root/dev/source-code:/root/dev/source-code/pro/python:$CCNET_CONF_DIR:$PYTHONPATH
TOP_DIR=$(python -c "import os; print(os.path.dirname(os.path.realpath('$0')))")
cd "$TOP_DIR"
python -m wsgidav.server.server_cli --host=0.0.0.0 --port=8080 --root=/ --server=gunicorn
chmod +x /root/dev/source-code/seafdav/run.sh
启动
screen -S seafdav
cd /root/dev/source-code/seafdav ./run.sh
最后回到主窗口。此时DAV服务在seafdav窗口内并发运行。
(推荐在最后完成此步骤)
安装ClamAV
groupadd clamav useradd -g clamav clamav sudo apt-get install clamav clamav-freshclam freshclam # 刷新病毒库,较慢,可暂时不用
推荐在此时启动上述所有需要启动的服务。
在确保seahub
、frontend
、seafevents
、seafes
、seafdav
都启动在各自窗口后,可以打开火狐浏览器,输入localhost:8000
进入网站进行测试。(用户名密码为此前设置的超级用户)
项目运行截图