FastDFS部署

最新推荐文章于 2024-08-01 17:52:41 发布

运维小王

最新推荐文章于 2024-08-01 17:52:41 发布

阅读量916

点赞数 34

分类专栏： Linux 文章标签： dfs linux

本文链接：https://blog.csdn.net/xiaowang0777/article/details/140696278

版权

Linux 专栏收录该内容

11 篇文章 2 订阅

订阅专栏

FastDFS部署

FastDFS 简介

Fastdfs(https://github.com/happyfish100) 是一个开源的高性能分布式文件系统（DFS）。它的主要功能包括：文件存储，文件同步和文件访问，以及高容量和负载平衡。主要解决了海量数据存储问题，特别适合以中小文件（建议范围：4KB < file_size <500MB）为载体的在线服务。
FastDFS 系统有三个角色：跟踪服务器(Tracker Server)和存储服务器(Storage Server)以及客户端(Client)。

Tracker Server： 跟踪服务器，主要做调度工作，起到均衡的作用；负责管理所有的 storage server和 group，每个 storage 在启动后会连接 Tracker，告知自己所属 group 等信息，并保持周期性心跳。

Storage Server： 存储服务器，主要提供容量和备份服务；以 group 为单位，每个 group 内可以有多台 storage server，数据互为备份。

Client：客户端： 上传下载数据的服务器，也就是我们自己的项目所部署在的服务器。

安装FastDFS

安装依赖

yum install -y git gcc gcc-c++ make automake autoconf libtool pcre pcre-devel zlib zlib-devel openssl-devel wget vim tree net-tools

下载软件包

wget https://github.com/happyfish100/libserverframe/archive/refs/tags/V1.2.4.tar.gz
wget https://github.com/happyfish100/libfastcommon/archive/refs/tags/V1.0.74.tar.gz
wget https://github.com/happyfish100/fastdfs/archive/refs/tags/V6.12.1.tar.gz

解压文件

tar -zxvf V1.2.4.tar.gz
tar -zxvf V1.0.74.tar.gz
tar -zxvf V6.12.1.tar.gz

编译FastDFS

# libfastcommon
cd libfastcommon-1.0.74
./make.sh && ./make.sh install

# libserverframe
cd libserverframe-1.2.4
./make.sh && ./make.sh install

# fastdfs
cd fastdfs-6.12.1
./make.sh && ./make.sh install

安装完毕，配置文件在/etc/fdfs 目录下；二进制程序在/usr/bin目录下；启动脚本在/Lib/systemd/system目录下。启动脚本要做一些修改设置完毕，还不能启动服务，因为我们还没有进行配置tracker服务器和storage服务器。所以接下来的工作就是配置tracker和storage服务。

Tracker服务端

配置Tracker服务

默认的配置文件为：

[root@node1 fdfs]# grep -v "^#" tracker.conf | grep -v "^$"
disabled = false
bind_addr =
port = 22122
address_family = auto
connect_timeout = 5
network_timeout = 60
base_path = /opt/fastdfs
......

根据实际情况和需要修改

# IP地址
bind_addr =

# 端口
port = 22122

# 文件位置
base_path = /opt/fastdfs

启动Trackerd进程

[root@node1 fdfs]# /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[root@node1 fdfs]# ps -ef | grep fdfs
root       4114      1  0 14:15 ?        00:00:00 /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
root       4145   1382  0 14:17 pts/0    00:00:00 grep --color=auto fdfs
[root@node1 fdfs]# netstat -natup | grep fdfs
tcp        0      0 192.168.30.101:22122    0.0.0.0:*               LISTEN      4114/fdfs_trackerd

服务启动后，会在 base_path 目录下创建 data、logs 两个目录。如下:

[root@node1 fdfs]# tree /opt/fastdfs/
/opt/fastdfs/
├── data
│   ├── fdfs_trackerd.pid
│   └── storage_changelog.dat
└── logs
    └── trackerd.log

2 directories, 3 files

Storage服务端

配置Storage服务

[root@node1 fdfs]# grep -v "^#" storage.conf | grep -v "^$"
disabled = false
group_name = group1
bind_addr =
client_bind = true
port = 23000
address_family = auto
connect_timeout = 5
network_timeout = 60
heart_beat_interval = 30
stat_report_interval = 60
base_path = /opt/fastdfs
max_connections = 1024
buff_size = 256KB
accept_threads = 1
work_threads = 4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec = 50
sync_interval = 0
sync_start_time = 00:00
sync_end_time = 23:59
write_mark_file_freq = 500
disk_recovery_threads = 3
store_path_count = 1
store_path0 = /opt/fastdfs
subdir_count_per_path = 256
tracker_server = 192.168.209.121:22122
......

根据实际情况和需要修改

# 组，使用默认即可
group_name = group1

# Storage所在服务器IP地址
bind_addr =

# 服务端口
port = 22122

# 文件位置 用于Storage存放日志
base_path = /opt/fastdfs

# 文件位置 用于Storage存储数据
store_path0 = /opt/fastdfs

# Tracker服务器IP和端口，不能是127.0.0.1
tracker_server = 192.168.209.121:22122

启动Storaged服务

启动服务

[root@node1 fdfs]# /usr/bin/fdfs_storaged /etc/fdfs/storage.conf start

查看日志

[root@node1 fdfs]# tail /opt/fastdfs/logs/storaged.log
mkdir data path: FA ...
mkdir data path: FB ...
mkdir data path: FC ...
mkdir data path: FD ...
mkdir data path: FE ...
mkdir data path: FF ...
data path: /opt/fastdfs/data, mkdir sub dir done.
[2024-07-25 14:26:25] INFO - file: storage_param_getter.c, line: 221, use_storage_id=0, id_type_in_filename=ip, trust_storage_server_id=1, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=20.00%, use_trunk_file=0, slot_min_size=256, slot_max_size=1024 KB, trunk_alloc_alignment_size=256, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_free_space_merge=1, delete_unused_trunk_files=0, trunk_compress_binlog_min_interval=86400, trunk_compress_binlog_interval=86400, trunk_compress_binlog_time_base=03:00, trunk_binlog_max_backups=7, store_slave_file_use_link=0
[2024-07-25 14:26:25] INFO - file: storage_func.c, line: 338, tracker_client_ip: 192.168.30.101, my_server_id_str: 192.168.30.101, g_server_id_in_filename: 1696508096
[2024-07-25 14:26:25] INFO - file: tracker_client_thread.c, line: 315, successfully connect to tracker server 192.168.30.101:22122, as a tracker client, my ip is 192.168.30.101

验证端口

[root@node1 fdfs]# netstat -natup | grep fdfs
tcp        0      0 192.168.30.101:22122    0.0.0.0:*               LISTEN      4114/fdfs_trackerd  
tcp        0      0 192.168.30.101:23000    0.0.0.0:*               LISTEN      4173/fdfs_storaged  
tcp        0      1 192.168.30.101:41155    192.168.209.121:22122   SYN_SENT    4163/fdfs_storaged  
tcp        0      0 192.168.30.101:49367    192.168.30.101:22122    ESTABLISHED 4173/fdfs_storaged  
tcp        0      0 192.168.30.101:22122    192.168.30.101:49367    ESTABLISHED 4114/fdfs_trackerd

Storage 服务启动后，在 base_path 下创建了 data、Logs 目录，记录着 Storage Server 的信息。在 store_pathθ 目录下，创建了 N*N 个子目录

[root@node1 fdfs]# ls /opt/fastdfs/data/
00  07  0E  15  1C  23  2A  31  38  3F  46  4D  54  5B  62  69  70  77  7E  85  8C  93  9A  A1  A8  AF  B6  BD  C4  CB  D2  D9  E0  E7  EE  F5  FC                     storage_groups_new.dat
01  08  0F  16  1D  24  2B  32  39  40  47  4E  55  5C  63  6A  71  78  7F  86  8D  94  9B  A2  A9  B0  B7  BE  C5  CC  D3  DA  E1  E8  EF  F6  FD                     storage_servers_new.dat
02  09  10  17  1E  25  2C  33  3A  41  48  4F  56  5D  64  6B  72  79  80  87  8E  95  9C  A3  AA  B1  B8  BF  C6  CD  D4  DB  E2  E9  F0  F7  fdfs_storaged.pid      storage_stat.dat
03  0A  11  18  1F  26  2D  34  3B  42  49  50  57  5E  65  6C  73  7A  81  88  8F  96  9D  A4  AB  B2  B9  C0  C7  CE  D5  DC  E3  EA  F1  F8  fdfs_trackerd.pid      storage_sync_timestamp.dat
04  0B  12  19  20  27  2E  35  3C  43  4A  51  58  5F  66  6D  74  7B  82  89  90  97  9E  A5  AC  B3  BA  C1  C8  CF  D6  DD  E4  EB  F2  F9  FE                     sync
05  0C  13  1A  21  28  2F  36  3D  44  4B  52  59  60  67  6E  75  7C  83  8A  91  98  9F  A6  AD  B4  BB  C2  C9  D0  D7  DE  E5  EC  F3  FA  FF
06  0D  14  1B  22  29  30  37  3E  45  4C  53  5A  61  68  6F  76  7D  84  8B  92  99  A0  A7  AE  B5  BC  C3  CA  D1  D8  DF  E6  ED  F4  FB  storage_changelog.dat

接下来查看Storage 服务与Tracker服务是否通信正常：

[root@node1 fdfs]# /usr/bin/fdfs_monitor /etc/fdfs/storage.conf
server_count=1, server_index=0

tracker server is 192.168.30.101:22122

group count: 1

Group 1:
group name = group1
......

文件上传测试

修改 Tracker服务器上的客户端配置文件，修改 base_path 及 tracker_server 配置，其他保持默认即可。如下:

[root@node1 fdfs]# grep -v "^#" client.conf | grep -v "^$"
connect_timeout = 5
network_timeout = 60
base_path = /opt/fastdfs
tracker_server = 192.168.0.196:22122
tracker_server = 192.168.0.197:22122
log_level = info
connect_first_by = tracker
use_connection_pool = false
connection_pool_max_idle_time = 3600
load_fdfs_parameters_from_tracker = false
use_storage_id = false
storage_ids_filename = storage_ids.conf
http.tracker_server_port = 80

根据实际情况和需要修改

# Tracker服务器IP和端口，不能是127.0.0.1
tracker_server = 192.168.209.121:22122

上传测试，执行如下命令：

[root@node1 fdfs]# fdfs_upload_file /etc/fdfs/client.conf haiyang.png 
group1/M00/00/00/wKgeZWah8t6AeYu1AAAAAAAAAAA855.png

上传成功后返回文件ID号：group1/M00/00/00/wKgeZWah8t6AeYu1AAAAAAAAAAA855.png。返回的文件ID由group、存储目录、两级子目录、fileid

安装Nginx服务

下载软件包

wget https://nginx.org/download/nginx-1.24.0.tar.gz

解压文件

tar -zxvf nginx-1.24.0.tar.gz

编译Nginx

[root@node1 ~]# cd nginx-1.24.0
[root@node1 nginx-1.24.0]# ./configure --prefix=/opt/module/nginx-1.24.0
[root@node1 nginx-1.24.0]# make && make install

验证Nginx

[root@node1 nginx-1.24.0]# /opt/module/nginx-1.24.0/sbin/nginx -V
nginx version: nginx/1.24.0
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
configure arguments: --prefix=/opt/module/nginx-1.24.0

配置Nginx

简单配置Nginx，可以使我们可以访问刚刚上传的文件。配置文件中添加一个／group1/Moθ 的location 即可。配置如下:

server {
	listen       80;
	location /group1/M00 {
		alias /opt/fastdfs/data;
	}
}

验证配置文件

[root@node1 conf]# /opt/module/nginx-1.24.0/sbin/nginx -t
nginx: the configuration file /opt/module/nginx-1.24.0/conf/nginx.conf syntax is ok
nginx: configuration file /opt/module/nginx-1.24.0/conf/nginx.conf test is successful

指定配置文件

[root@node1 conf]# /opt/module/nginx-1.24.0/sbin/nginx -c /opt/module/nginx-1.24.0/conf/nginx.conf

验证端口

[root@node1 conf]# ps -ef | grep nginx
root       6849      1  0 14:57 ?        00:00:00 nginx: master process /opt/module/nginx-1.24.0/sbin/nginx -c /opt/module/nginx-1.24.0/conf/nginx.conf
nobody     6850   6849  0 14:57 ?        00:00:00 nginx: worker process
root       6854   1382  0 14:58 pts/0    00:00:00 grep --color=auto nginx

打开浏览器访问刚刚上传的文件，URL为：http://192.168.30.101/group1/M00/00/00/wKgeZWah8t6AeYu1AAAAAAAAAAA855.png

使用命令行curl访问

[root@node1 conf]# curl -I http://192.168.30.101/group1/M00/00/00/wKgeZWah8t6AeYu1AAAAAAAAAAA855.png
HTTP/1.1 200 OK
Server: nginx/1.24.0
Date: Thu, 25 Jul 2024 07:00:43 GMT
Content-Type: image/png
Content-Length: 0
Last-Modified: Thu, 25 Jul 2024 06:38:22 GMT
Connection: keep-alive
ETag: "66a1f2de-0"
Accept-Ranges: bytes

Nginx集成FastDFS模块

下载软件包

wget https://github.com/happyfish100/fastdfs-nginx-module/archive/refs/tags/V1.24.tar.gz

解压文件

tar -zxvf V1.24.tar.gz

重新编译Nginx

停止Nginx服务

[root@node1 ~]# /opt/module/nginx-1.24.0/sbin/nginx -s stop

编译安装Nginx

[root@node1 ~]# cd nginx-1.24.0
[root@node1 nginx-1.24.0]# ./configure --prefix=/opt/module/nginx-1.24.0 --add-module=/root/fastdfs-nginx-module-1.24/src

[root@node1 nginx-1.24.0]# make && make install

查看Nginx版本

[root@node1 nginx-1.24.0]# /opt/module/nginx-1.24.0/sbin/nginx -V
nginx version: nginx/1.24.0
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
configure arguments: --prefix=/opt/module/nginx-1.24.0 --add-module=/root/fastdfs-nginx-module-1.24/src

配置Nginx

复制fastdfs-nginx-module 源码中的配置文件到/etc/fdfs 目录，并修改:

[root@node1 ~]# cd fastdfs-nginx-module-1.24/
[root@node1 fastdfs-nginx-module-1.24]# ls src/
common.c  common.h  config  mod_fastdfs.conf  ngx_http_fastdfs_module.c
[root@node1 fastdfs-nginx-module-1.24]# cp ./src/mod_fastdfs.conf /etc/fdfs/

修改如下：

# 连接超时时间
connect_timeout=10

# Tracker Server
tracker_server=192.168.30.101:22122

# StorageServer默认端口
storage_server_port=23000

修改Nginx配置

server {
	listen       80;
	location /group([0-9])/M00 {
		ngx_fastdfs_module;
	}
}

启动Nginx

[root@node1 ~]# /opt/module/nginx-1.24.0/sbin/nginx -t
ngx_http_fastdfs_set pid=9517
nginx: the configuration file /opt/module/nginx-1.24.0/conf/nginx.conf syntax is ok
nginx: configuration file /opt/module/nginx-1.24.0/conf/nginx.conf test is successful

[root@node1 ~]# /opt/module/nginx-1.24.0/sbin/nginx -c /opt/module/nginx-1.24.0/conf/nginx.conf
ngx_http_fastdfs_set pid=9518

[root@node1 ~]# ps -ef | grep nginx
root       9519      1  0 15:20 ?        00:00:00 nginx: master process /opt/module/nginx-1.24.0/sbin/nginx -c /opt/module/nginx-1.24.0/conf/nginx.conf
root       9522   1382  0 15:20 pts/0    00:00:00 grep --color=auto nginx

开机自启

FsatDFS开机自启

修改文件内的路径(默认安装不需要修改)

[root@node1 ~]# cd /usr/lib/systemd/system
[root@node1 system]# ls | grep fdfs
fdfs_storaged.service
fdfs_trackerd.service

配置开机自启

[root@node1 system]# systemctl enable fdfs_trackerd
Created symlink from /etc/systemd/system/multi-user.target.wants/fdfs_trackerd.service to /usr/lib/systemd/system/fdfs_trackerd.service.
[root@node1 system]# systemctl enable fdfs_storaged
Created symlink from /etc/systemd/system/multi-user.target.wants/fdfs_storaged.service to /usr/lib/systemd/system/fdfs_storaged.service.

Nginx开机自启

编辑文件

vim /usr/lib/systemd/system/nginx.service

[Unit]
Description=nginx - high performance web server
Documentation=http://nginx.org/en/docs/
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target
[Service]
Type=forking
PIDFile=/opt/module/nginx-1.24.0/logs/nginx.pid
ExecStart=/opt/module/nginx-1.24.0/nginx -c /opt/module/nginx-1.24.0/conf/nginx.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
[Install]
WantedBy=multi-user.target

配置开机自启

[root@node1 system]# systemctl daemon-reload
[root@node1 system]# systemctl enable nginx
Created symlink from /etc/systemd/system/multi-user.target.wants/nginx.service to /usr/lib/systemd/system/nginx.service.