FastDFS高可用集群架构配置搭建实战。有问题欢迎指出。注意按照此文档版本少采坑

最新推荐文章于 2021-07-25 17:38:03 发布

wo1901446409

最新推荐文章于 2021-07-25 17:38:03 发布

阅读量452

点赞数

分类专栏： fastdfs 文章标签： fastdfs 分布式文件

本文链接：https://blog.csdn.net/wo1901446409/article/details/103511534

版权

fastdfs 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

注意版本问题。真特么坑

FastDfs

wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz

libfastcommon

wget https://github.com/happyfish100/libfastcommon/archive/V1.0.39.tar.gz

fastdfs-nginx-module

wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz

nginx

wget http://nginx.org/download/nginx-1.12.2.tar.gz

集群准备

	服务所需要安装的软件
Vip 192.168.10.50	nginx + keepalived	192.168.xx.106
Storage2	FastDFS,libfastcommon,nginx,fastdfs-nginx-module	192.168.xx.107
Tracker1	FastDFS,libfastcommon	192.168.xx.108
Tracker2	FastDFS,libfastcommon	192.168.xx.109
Storage1	FastDFS,libfastcommon,nginx,fastdfs-nginx-module	192.168.xx.110
Vip 192.168.10.50	nginx + keepalived	192.168.xx.111

所有安装包	/opt/fastdfs
数据存储位置	/data/fastdfs

1、编译环境准备

yum install git gcc gcc-c++ make automake autoconf libtool pcre pcre-devel zlib zlib-devel openssl-devel wget vim -y

$ mkdir -p /opt/fastdfs /data/fastdfs

$ cd /opt/fastdfs #为下一步下载源码做准备

2、安装libfastcommon

#下载文件

$ wget https://github.com/happyfish100/libfastcommon/archive/V1.0.39.tar.gz

$ tar -zxvf libfastcommon-1.0.39.tar.gz

$ cd libfastcommon-1.0.39/

$ ./make.sh

$ ./make.sh install

3、安装FastDFS

$ wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz

$ tar -zxvf fastdfs-5.11.tar.gz

$ cd fastdfs-5.11/

$ ./make.sh

$ ./make.sh install

#配置文件准备

$ cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf #tracker节点

$ cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf #storage节点

$ cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf #客户端文件，测试用（tracker节点做测试用）

$ cp /opt/fastdfs/fastdfs-5.11/conf/http.conf /etc/fdfs/ #供nginx访问使用

$ cp /opt/fastdfs/fastdfs-5.11/conf/mime.types /etc/fdfs/ #供nginx访问使用

tracker server配置:

$ vim /etc/fdfs/tracker.conf

#需要修改的内容如下

port=22122 # tracker服务器端口（默认22122,一般不修改）

base_path=/data/fastdfs # 存储日志和数据的根目录

#编辑启动文件

$ vim /usr/lib/systemd/system/fastdfs-tracker.service

[Unit]

Description=The FastDFS File server

After=network.target remote-fs.target nss-lookup.target

[Service]

Type=forking

ExecStart=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf start

ExecStop=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf stop

ExecRestart=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart

[Install]

WantedBy=multi-user.target

$ systemctl daemon-reload

$ systemctl enable fastdfs-tracker.service

$ systemctl start fastdfs-tracker.service

$ netstat -tulnp #查看服务是否启动，端口是否打开

storage server配置

$ vim /etc/fdfs/storage.conf

#需要修改的内容如下

port=23000 # storage服务端口（默认23000,一般不修改）

base_path=/data/fastdfs # 数据和日志文件存储根目录

store_path0=/data/fastdfs # 第一个存储目录

tracker_server=192.168.xx.108:22122 # tracker服务器IP和端口

tracker_server=192.168.xx.109:22122 # tracker服务器IP和端口

http.server_port=8888 # http访问文件的端口(默认8888,看情况修改,和nginx中保持一致)

#编辑启动文件

$ vim /usr/lib/systemd/system/fastdfs-storage.service

[Unit]

Description=The FastDFS File server

After=network.target remote-fs.target nss-lookup.target

[Service]

Type=forking

ExecStart=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf start

ExecStop=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf stop

ExecRestart=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart

[Install]

WantedBy=multi-user.target

$ systemctl daemon-reload

$ systemctl enable fastdfs-storage.service

$ systemctl start fastdfs-storage.service

$ netstat -tulnp #查看服务是否启动，端口是否打开

#查看集群状态

$ fdfs_monitor /etc/fdfs/storage.conf list

Client配置

$ vim /etc/fdfs/client.conf

#需要修改的内容如下

base_path=/data/fastdfs

tracker_server=192.168.xx.108:22122 # tracker服务器IP和端口

tracker_server=192.168.xx.109:22122 # tracker服务器IP和端口

#保存后测试,返回ID表示成功 如：group1/M00/00/00/xx.tar.gz

$ fdfs_upload_file /etc/fdfs/client.conf testfile

$ fdfs_upload_file /etc/fdfs/client.conf  css.jpg

group2/M00/00/00/wKgKa13x4g6AFoZQAABO79zQokI079.jpg

4、安装nginx和fastdfs-nginx-module

下载nginx module、并解压

$ wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz

$ wget tar –zxvf V1.20.tar.gz

$ cp /opt/fastdfs/fastdfs-nginx-module-1.20/src/mod_fastdfs.conf /etc/fdfs

安装nginx ()

$ yum install nginx -y

# 查看版本及编译参数

$ nginx -v

$ nginx -V

添加nginx、 fast-nginx-module

$ wget http://nginx.org/download/nginx-1.12.2.tar.gz

$ tar -zxvf nginx-1.12.2.tar.gz

$ cd nginx-1.12.2/

把fast-nginx-module 添加到nginx (add-module 为fast模块src路径)

注意：configure之前要修改 /opt/fastdfs/fastdfs-nginx-module-1.20/src下的config

表红部分一定要改成自己的fastcommon 所在目录, 编辑后的效果如下

$vim config

ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon"

CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon"

$ ./configure --prefix=/usr/local/nginx --add-module= /opt/fastdfs/fastdfs-nginx-module-1.20/src

$make

$make install

编译之后的添加fast 模块的nginx 文件覆盖之前的nginx

cp objs/nginx /usr/sbin/nginx

$./nginx –v 查看nginx 信息。

配置nginx访问

$ vim /etc/fdfs/mod_fastdfs.conf

#需要修改的内容如下

tracker_server=192.168.xx.108:22122 # tracker服务器IP和端口

tracker_server=192.168.xx.109:22122 # tracker服务器IP和端口

url_have_group_name=true

base_path=/data/fastdfs

store_path0=/data/fastdfs

#配置nginx.config

$ vim /etc/nginx/nginx.conf

#增加如下内容

server {

listen 8888; ## 该端口为storage.conf中的http.server_port相同

server_name localhost;

location ~/group[0-9]/ {

root /data/fastdfs;

ngx_fastdfs_module;

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root html;

}

创建软连接根据自己的data实现目录

ln -s data/fastdfs/data/ data/fastdfs/data/M00

#测试下载，用外部浏览器访问刚才client已传过的文件,引用返回的ID

http://192.168.10.110:8888/group1/M00/00/00/wKgKbl3vVJSAUdB0AAEs0qb8nEI235.jpg

根据上传图片M00/后的路径查看文件。

5、配置文件访问的负载均衡和高可用

在192.168.10.106和111上安装nginx、keepalived

yum install -y nginx keepalived

xx.106 的nginx的配置文件如下：

server {

listen 80;

server_name wms.jxdd.com;（此电脑非干净机器，无奈只能把自己的配置添加到server 里面）

location /group1/M00 {

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_pass http://fdfs_group1;

}

location /group2/M00 {

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_pass http://fdfs_group2;

}

location / {

root html/wms;

index index.html index.htm;

}

Upstream 应该配置nginx.conf http 里面。必须在server外面

upstream fdfs_group1 {

server 192.168.10.xxx:8888 weight=1 max_fails=2 fail_timeout=30s;

# server 192.168.10.xxx:8888 weight=1 max_fails=2 fail_timeout=30s;多个的话，可以配置多个

}

upstream fdfs_group2 {

server 192.168.10.107:8888 weight=1 max_fails=2 fail_timeout=30s;

}

xx.106 keepalived.conf 配置文件修改

xx.106 为nginx 高可用主节点，xx.111 是备用节点。注意virtual_router_id 要一样，主节点的priority 要不被备用节点的priority 大

global_defs {

router_id edu-proxy-01

}

vrrp_script chk_nginx {

script "/etc/keepalived/check_nginx.sh"

interval 2

weight -20

}

vrrp_instance VI_1 {

state MASTER

interface eth0 #根据自己的网卡设置

virtual_router_id 51

mcast_src_ip 192.168.xx.106#本机ip

priority 100

nopreempt

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

track_script {

chk_nginx

}

virtual_ipaddress {

192.168.10.50#vip 虚拟地址

}

Check 心跳检查文件。。 keepalived.conf 中配置用的。

cd /etc/fdfs

vim check_nginx.sh

#!/bin/bash

counter=$(ps -C nginx --no-heading|wc -l)
if [ "${counter}" = "0" ]; then
    exit 1
else
    exit 0
fi

Check_nginx.sh创建成功后执行脚本让其 作为可执行程序

$chmod +x ./ check_nginx.sh

xx.111 的nginx的配置文件如下

server {

listen 80;

server_name pms.jxdd.com; #站点ip

location /group1/M00 {

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_pass http://fdfs_group1;

}

location /group2/M00 {

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_pass http://fdfs_group2;

}

location / {

root html/pms; #项目文件地址

index index.html;

}

upstream fdfs_group1 {

server 192.168.xx.110:8888 weight=1 max_fails=2 fail_timeout=30s;

}

upstream fdfs_group2 {

server 192.168.xx.107:8888 weight=1 max_fails=2 fail_timeout=30s;

}

xx.111 keepalived.conf 配置文件修改

! Configuration File for keepalived

global_defs {

router_id edu-proxy-02

}

vrrp_script chk_nginx {

script "/etc/keepalived/check_nginx.sh"

interval 2

weight -20

}

vrrp_instance VI_1 {

state BACKUP

interface eth0

virtual_router_id 51

mcast_src_ip 192.168.xx.111 #本机ip

priority 90

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

track_script {

chk_nginx

}

virtual_ipaddress {

192.168.10.50# vip 虚拟路径

}

nginx、keepalived都配置好了后，先启动nginx，再启动keepalived

最终的效果的

http://192.168.10.50//group1/M00/00/00/wKgKbl3xtsuALnUvAAFE9Seg04Y366.jpg

参考文章：

https://www.cnblogs.com/sunnydou/p/49b92d511047f4f9da6cd727cfd415d5.html

https://www.cnblogs.com/youzhibing/p/9187765.html

ln -fs /usr/local/lib/libfastcommon.so.1 /usr/local/lib/libfastcommon.so

ln -fs /usr/local/lib/libfastcommon.so.1 /usr/lib/libfastcommon.so

scp -r fastdfs root@192.168.xx.xx:/opt

把一个目录拷贝到其他服务器

/etc/fdfs 配置目录

Keepalived配置文件目录

/etc/keepalived

/var/logs/messages(是keepalived的日志)

1.注意删掉message 之后必须得重启rsyslog。否则日志无信息

centos7重启rsyslog服务：

systemctl restart rsyslog

使用：（killall无效）

killall -HUP rsyslog

nginx配置文件目录

/usr/local/nginx

/usr/local/nginx/logs error.log 目录

/usr/local/nginx/conf nginx.conf 目录

/usr/local/nginx/sbin 启动命令目录

注意：如果服务器有防火墙，自己再防火墙添加开放各个端口。如果是阿里云的话，自己配置各个安全组。

查看防火墙

yum list installed | grep firewalld

命令：启动nginx

Usr/local/nginx/sbin/./nginx

重启nginx

Usr/local/nginx/sbin/./nginx –s reload

启动知道配置文件

Usr/local/nginx/sbin/./nginx –c/opt/fastdfs/nginx-1.12/conf/nginx.conf

/usr/local/nginx/sbin/nginx #启动nginx

/usr/local/nginx/sbin/nginx -s reload #重启nginx

/usr/local/nginx/sbin/nginx -s stop #停止nginx

清空文件内容

cat /null > messages

查看ip 网卡信息

ifconfig

开启keepalived

/bin/systemctl start keepalived.service

查看是否起来。

Ps –ef | grep keepalived

停止keepalived

/bin/systemctl stop keepalived.service

Keepalived日志默认 var/logs/message

配置文件 /etc/keepalived/…

修改keepalived 日志参考下文档

https://www.cnblogs.com/hwlong/p/6070575.html

tracker.conf的配置文件

# the method of selecting group to upload files

# 0: round robin

# 1: specify group

# 2: load balance, select the max free space group to upload file

store_lookup=2

＃选择上传文件组的方法
＃0：循环赛
＃1：指定组
＃2：负载均衡，选择最大可用空间组来上传文件

wo1901446409

关注

0
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
FastDFS高可用集群架构配置搭建实战。有问题欢迎指出。注意按照此文档版本少采坑

注意版本问题。真特么坑FastDfswget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gzlibfastcommonwget https://github.com/happyfish100/libfastcommon/archive/V1.0.39.tar.gzfastdfs-nginx-modul...
复制链接

扫一扫