注意版本问题。真特么坑
FastDfs
wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz
libfastcommon
wget https://github.com/happyfish100/libfastcommon/archive/V1.0.39.tar.gz
fastdfs-nginx-module
wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz
nginx
wget http://nginx.org/download/nginx-1.12.2.tar.gz
集群准备
| 服务所需要安装的软件 |
|
Vip 192.168.10.50 | nginx + keepalived | 192.168.xx.106 |
Storage2 | FastDFS,libfastcommon,nginx,fastdfs-nginx-module | 192.168.xx.107 |
Tracker1 | FastDFS,libfastcommon | 192.168.xx.108 |
Tracker2 | FastDFS,libfastcommon | 192.168.xx.109 |
Storage1 | FastDFS,libfastcommon,nginx,fastdfs-nginx-module | 192.168.xx.110 |
Vip 192.168.10.50 | nginx + keepalived | 192.168.xx.111 |
所有安装包 | /opt/fastdfs |
数据存储位置 | /data/fastdfs |
1、编译环境准备
yum install git gcc gcc-c++ make automake autoconf libtool pcre pcre-devel zlib zlib-devel openssl-devel wget vim -y
创建安装包目录、进入目录下载 安装包
$ mkdir -p /opt/fastdfs /data/fastdfs
$ cd /opt/fastdfs #为下一步下载源码做准备
2、安装libfastcommon
#下载文件
$ wget https://github.com/happyfish100/libfastcommon/archive/V1.0.39.tar.gz
$ tar -zxvf libfastcommon-1.0.39.tar.gz
$ cd libfastcommon-1.0.39/
$ ./make.sh
$ ./make.sh install
3、安装FastDFS
$ wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz
$ tar -zxvf fastdfs-5.11.tar.gz
$ cd fastdfs-5.11/
$ ./make.sh
$ ./make.sh install
#配置文件准备
$ cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf #tracker节点
$ cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf #storage节点
$ cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf #客户端文件,测试用(tracker节点做测试用)
$ cp /opt/fastdfs/fastdfs-5.11/conf/http.conf /etc/fdfs/ #供nginx访问使用
$ cp /opt/fastdfs/fastdfs-5.11/conf/mime.types /etc/fdfs/ #供nginx访问使用
tracker server配置:
$ vim /etc/fdfs/tracker.conf
#需要修改的内容如下
port=22122 # tracker服务器端口(默认22122,一般不修改)
base_path=/data/fastdfs # 存储日志和数据的根目录
#编辑启动文件
$ vim /usr/lib/systemd/system/fastdfs-tracker.service
[Unit]
Description=The FastDFS File server
After=network.target remote-fs.target nss-lookup.target
[Service]
Type=forking
ExecStart=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf start
ExecStop=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf stop
ExecRestart=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
[Install]
WantedBy=multi-user.target
$ systemctl daemon-reload
$ systemctl enable fastdfs-tracker.service
$ systemctl start fastdfs-tracker.service
$ netstat -tulnp #查看服务是否启动,端口是否打开
storage server配置
$ vim /etc/fdfs/storage.conf
#需要修改的内容如下
port=23000 # storage服务端口(默认23000,一般不修改)
base_path=/data/fastdfs # 数据和日志文件存储根目录
store_path0=/data/fastdfs # 第一个存储目录
tracker_server=192.168.xx.108:22122 # tracker服务器IP和端口
tracker_server=192.168.xx.109:22122 # tracker服务器IP和端口
http.server_port=8888 # http访问文件的端口(默认8888,看情况修改,和nginx中保持一致)
#编辑启动文件
$ vim /usr/lib/systemd/system/fastdfs-storage.service
[Unit]
Description=The FastDFS File server
After=network.target remote-fs.target nss-lookup.target
[Service]
Type=forking
ExecStart=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf start
ExecStop=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf stop
ExecRestart=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
[Install]
WantedBy=multi-user.target
$ systemctl daemon-reload
$ systemctl enable fastdfs-storage.service
$ systemctl start fastdfs-storage.service
$ netstat -tulnp #查看服务是否启动,端口是否打开
#查看集群状态
$ fdfs_monitor /etc/fdfs/storage.conf list
Client配置
$ vim /etc/fdfs/client.conf
#需要修改的内容如下
base_path=/data/fastdfs
tracker_server=192.168.xx.108:22122 # tracker服务器IP和端口
tracker_server=192.168.xx.109:22122 # tracker服务器IP和端口
#保存后测试,返回ID表示成功 如:group1/M00/00/00/xx.tar.gz
$ fdfs_upload_file /etc/fdfs/client.conf testfile
$ fdfs_upload_file /etc/fdfs/client.conf css.jpg
group2/M00/00/00/wKgKa13x4g6AFoZQAABO79zQokI079.jpg
4、安装nginx和fastdfs-nginx-module
下载nginx module、并解压
$ wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz
$ wget tar –zxvf V1.20.tar.gz
$ cp /opt/fastdfs/fastdfs-nginx-module-1.20/src/mod_fastdfs.conf /etc/fdfs
安装nginx ()
$ yum install nginx -y
# 查看版本及编译参数
$ nginx -v
$ nginx -V
添加nginx、 fast-nginx-module
$ wget http://nginx.org/download/nginx-1.12.2.tar.gz
$ tar -zxvf nginx-1.12.2.tar.gz
$ cd nginx-1.12.2/
把fast-nginx-module 添加到nginx (add-module 为fast模块src路径)
注意:configure之前要修改 /opt/fastdfs/fastdfs-nginx-module-1.20/src下的config
表红部分一定要改成自己 的fastcommon 所在目录, 编辑后的效果 如下
$vim config
ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon"
CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon"
$ ./configure --prefix=/usr/local/nginx --add-module= /opt/fastdfs/fastdfs-nginx-module-1.20/src
$make
$make install
编译之后的添加fast 模块的nginx 文件 覆盖之前的nginx
cp objs/nginx /usr/sbin/nginx
$./nginx –v 查看nginx 信息。
配置nginx访问
$ vim /etc/fdfs/mod_fastdfs.conf
#需要修改的内容如下
tracker_server=192.168.xx.108:22122 # tracker服务器IP和端口
tracker_server=192.168.xx.109:22122 # tracker服务器IP和端口
url_have_group_name=true
base_path=/data/fastdfs
store_path0=/data/fastdfs
#配置nginx.config
$ vim /etc/nginx/nginx.conf
#增加如下内容
server {
listen 8888; ## 该端口为storage.conf中的http.server_port相同
server_name localhost;
location ~/group[0-9]/ {
root /data/fastdfs;
ngx_fastdfs_module;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
创建软连接 根据自己的data实现目录
ln -s data/fastdfs/data/ data/fastdfs/data/M00
#测试下载,用外部浏览器访问刚才client已传过的文件,引用返回的ID
http://192.168.10.110:8888/group1/M00/00/00/wKgKbl3vVJSAUdB0AAEs0qb8nEI235.jpg
根据上传图片M00/后的路径查看文件。
5、配置文件访问的负载均衡和高可用
在192.168.10.106和111上安装nginx、keepalived
yum install -y nginx keepalived
xx.106 的nginx的配置文件如下:
server {
listen 80;
server_name wms.jxdd.com;(此电脑非干净机器,无奈只能把自己的配置添加到server 里面)
location /group1/M00 {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://fdfs_group1;
}
location /group2/M00 {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://fdfs_group2;
}
location / {
root html/wms;
index index.html index.htm;
}
}
Upstream 应该配置nginx.conf http 里面。必须在server外面
upstream fdfs_group1 {
server 192.168.10.xxx:8888 weight=1 max_fails=2 fail_timeout=30s;
# server 192.168.10.xxx:8888 weight=1 max_fails=2 fail_timeout=30s;多个的话,可以配置多个
}
upstream fdfs_group2 {
server 192.168.10.107:8888 weight=1 max_fails=2 fail_timeout=30s;
}
xx.106 keepalived.conf 配置文件修改
xx.106 为nginx 高可用主节点,xx.111 是备用节点。注意virtual_router_id 要一样,主节点的priority 要不被备用节点的priority 大
global_defs {
router_id edu-proxy-01
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_nginx.sh"
interval 2
weight -20
}
vrrp_instance VI_1 {
state MASTER
interface eth0 #根据自己的网卡设置
virtual_router_id 51
mcast_src_ip 192.168.xx.106#本机ip
priority 100
nopreempt
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_nginx
}
virtual_ipaddress {
192.168.10.50#vip 虚拟地址
}
Check 心跳检查文件。。 keepalived.conf 中配置用的。
cd /etc/fdfs
vim check_nginx.sh
#!/bin/bash
counter=$(ps -C nginx --no-heading|wc -l)
if [ "${counter}" = "0" ]; then
exit 1
else
exit 0
fi
Check_nginx.sh创建成功后执行脚本让其 作为可执行程序
$chmod +x ./ check_nginx.sh
xx.111 的nginx的配置文件如下
server {
listen 80;
server_name pms.jxdd.com; #站点ip
location /group1/M00 {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://fdfs_group1;
}
location /group2/M00 {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://fdfs_group2;
}
location / {
root html/pms; #项目文件地址
index index.html;
}
}
upstream fdfs_group1 {
server 192.168.xx.110:8888 weight=1 max_fails=2 fail_timeout=30s;
}
upstream fdfs_group2 {
server 192.168.xx.107:8888 weight=1 max_fails=2 fail_timeout=30s;
}
xx.111 keepalived.conf 配置文件修改
! Configuration File for keepalived
global_defs {
router_id edu-proxy-02
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_nginx.sh"
interval 2
weight -20
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
mcast_src_ip 192.168.xx.111 #本机ip
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_nginx
}
virtual_ipaddress {
192.168.10.50# vip 虚拟路径
}
}
nginx、keepalived都配置好了后,先启动nginx,再启动keepalived
最终的效果的
http://192.168.10.50//group1/M00/00/00/wKgKbl3xtsuALnUvAAFE9Seg04Y366.jpg
参考文章:
https://www.cnblogs.com/sunnydou/p/49b92d511047f4f9da6cd727cfd415d5.html
https://www.cnblogs.com/youzhibing/p/9187765.html
ln -fs /usr/local/lib/libfastcommon.so.1 /usr/local/lib/libfastcommon.so
ln -fs /usr/local/lib/libfastcommon.so.1 /usr/lib/libfastcommon.so
scp -r fastdfs root@192.168.xx.xx:/opt
把一个目录拷贝到 其他服务器
目录
/etc/fdfs 配置目录
Keepalived配置文件目录
/etc/keepalived
/var/logs/messages(是keepalived的日志)
1.注意 删掉message 之后 必须得重启rsyslog。否则日志无信息
centos7重启rsyslog服务:
systemctl restart rsyslog
使用:(killall无效)
killall -HUP rsyslog
nginx配置文件目录
/usr/local/nginx
/usr/local/nginx/logs error.log 目录
/usr/local/nginx/conf nginx.conf 目录
/usr/local/nginx/sbin 启动命令目录
注意:如果服务器有防火墙,自己再防火墙 添加 开放各个端口。如果是阿里云的话,自己配置各个安全组。
查看防火墙
yum list installed | grep firewalld
命令:启动nginx
Usr/local/nginx/sbin/./nginx
重启nginx
Usr/local/nginx/sbin/./nginx –s reload
启动知道配置文件
Usr/local/nginx/sbin/./nginx –c/opt/fastdfs/nginx-1.12/conf/nginx.conf
/usr/local/nginx/sbin/nginx #启动nginx
/usr/local/nginx/sbin/nginx -s reload #重启nginx
/usr/local/nginx/sbin/nginx -s stop #停止nginx
清空文件内容
cat /null > messages
查看ip 网卡信息
ifconfig
开启keepalived
/bin/systemctl start keepalived.service
查看是否起来。
Ps –ef | grep keepalived
停止keepalived
/bin/systemctl stop keepalived.service
Keepalived日志默认 var/logs/message
配置文件 /etc/keepalived/…
修改keepalived 日志参考下文档
https://www.cnblogs.com/hwlong/p/6070575.html
tracker.conf的配置文件
# the method of selecting group to upload files
# 0: round robin
# 1: specify group
# 2: load balance, select the max free space group to upload file
store_lookup=2
#选择上传文件组的方法
#0:循环赛
#1:指定组
#2:负载均衡,选择最大可用空间组来上传文件