分布式文件系统FastDFS
文件系统概述
分布式文件系统（Distributed File System）是指文件系统管理的物理存储资源不一定直接连接在本地节点上，而是通过计算机网络与节点相连。
文件系统分类
①块存储：MongoDB数据库中的GridFS、Hadoop中的HDFS，这些系统在存储文件的的时候会尝试先将文件打碎存储（拆分成Data Block）。这样存储的优点可以存储超大型文件，更加高效的利用磁盘资源。但是需要额外存储文件碎片的元数据信息。
②文件存储:GlusterFS、NFS、FastDFS等都是基于文件单位存储，这种存储并不会将文件系统打碎。而是文件存储到系统中的某一台服务器中。这样存储的优点可以应对一些小文件系统，系统维护简单，无需存储文件的元数据，系统设计和维护成本低。
FastDFS存储原理
FastDFS中Storage的卷是一样的也就意味着这些服务彼此数据相互备份，实现数据的冗余备份。不同卷存储整个集群中的部分文件，类似于传统单机文件系统的的分区概念（C盘、D盘、…）Tracker Server 主要做工作调度，在访问上起负载均衡的作用。在内存中记录集群中group/卷和Storage server的状态信息，是连接Client和Storage Server的枢纽。因为相关信息存储在内存中，所以Tracker Server性能非常高,一个较大的集群（上百个group/卷）中3台就够了。
项目中我们使用FastDFS存储小文件，例如图片，音频和视频等。
上传、下载机制
Storage Server会定期的向Tracker服务器汇报自身状态信息，例如健康状态和存储容量。
Client连接Tracker Server发送文件请求。
Tracker Server根据注册的Storage Server的信息返回一台可用的Storage Server的调用信息。
Client拿到信息后直接连接对应的Storage Server进行点到点的文件上传（附加元数据-可选）。
Storage Server收到文件请求后会根据自己位置信息生成File_ID信息，并且将File_ID和用户携带的元数据信息进行关联，然后将File_ID返回给Client。
返回的File_ID文件服务器并不会存储，需要Client端保留在外围数据库中，以后Client端可以通过File_ID下载对应的文件或者元数据。
Storage Server会定期的向Tracker服务器汇报自身状态信息，例如健康状态和存储容量。
Client连接Tracker Server携带File_ID参数发送文件下载请求。
Tracker Server根据注册的Storage Server的信息返回一台可用的Storage Server的调用信息(主从服务器)。
Client拿到信息后直接连接对应的Storage Server进行点到点的文件下载（读取元数据）。
Storage Server收到文件请求后解析File_ID信息，读取本地的文件流，将数据写会Client端。
File_ID组成
File_ID是由Storage Server生成并返回给Client端。File_ID包含了组/卷和文件路径。Storage Server可以直接根据该文件名定位到该文件。
FastDFS集群搭建
同步集群机器时间
[root@centos ~]# yum -y install ntp ntpdate
[root@centos ~]# ntpdate cn.pool.ntp.org
资源下载安装
进入 https://github.com/happyfish100 网站下载FastDFS相关资源。
1.安装依赖包libfastcommon
https://github.com/happyfish100/libfastcommon/archive/V1.0.39.tar.gz
[root@CentOSX ~]# yum install -y gcc-c++
[root@CentOSX ~]# tar -zxf libfastcommon-1.0.39.tar.gz
[root@CentOSX ~]# cd libfastcommon-1.0.39
[root@CentOSX libfastcommon-1.0.39]# ./make.sh
[root@CentOSX libfastcommon-1.0.39]# ./make.sh install
2.安装FastDFS
https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz
[root@CentOSX ~]# yum install -y perl-devel
[root@CentOSX ~]# tar -zxf fastdfs-5.11.tar.gz
[root@CentOSX ~]# cd fastdfs-5.11
[root@CentOSX fastdfs-5.11]# ./make.sh
[root@CentOSX fastdfs-5.11]# ./make.sh install
提示:当软件安装结束后，默认FastDFS启动所需的配置文件放置在/etc/fdfs目录下。
[root@CentOSX ~]# yum install -y tree
[root@CentOSX ~]# tree /etc/fdfs/
/etc/fdfs/
├── client.conf.sample
├── storage.conf.sample
├── storage_ids.conf.sample
└── tracker.conf.sample
0 directories, 4 files
# 可运行脚本
[root@CentOSX ~]# ls -l /etc/init.d/fdfs_*
-rwxr-xr-x. 1 root root 961 Jun 30 20:22 /etc/init.d/fdfs_storaged
-rwxr-xr-x. 1 root root 963 Jun 30 20:22 /etc/init.d/fdfs_trackerd
# 执行程序
[root@CentOSX ~]# whereis fdfs_storaged fdfs_trackerd
fdfs_storaged: /usr/bin/fdfs_storaged
fdfs_trackerd: /usr/bin/fdfs_trackerd
配置服务
- 1.创建fdfs运行所需的数据目录
[root@CentOSX ~]# mkdir -p /data/fdfs/{tracker,storage/store01,storage/store02}
[root@CentOSX ~]# tree /data/
/data/
└── fdfs
├── storage
│ └── store01
| └── store02
└── tracker
- 2.创建启动所需的配置文件
[root@CentOSX ~]# cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf
[root@CentOSX ~]# cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf
[root@CentOSX ~]# cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
[root@CentOSX ~]# tree /etc/fdfs/
/etc/fdfs/
├── client.conf
├── client.conf.sample
├── storage.conf
├── storage.conf.sample
├── storage_ids.conf.sample
├── tracker.conf
└── tracker.conf.sample
- 3.配置Tracker Server
[root@CentOSX ~]# vi /etc/fdfs/tracker.conf
base_path=/data/fdfs/tracker
- 4.配置Storage Server
[root@CentOSX ~]# vi /etc/fdfs/storage.conf
base_path=/data/fdfs/storage
store_path_count=2
store_path0=/data/fdfs/storage/store01
store_path1=/data/fdfs/storage/store02
tracker_server=CentOSA:22122
tracker_server=CentOSB:22122
tracker_server=CentOSC:22122
- 5.修改Client端
[root@CentOSX ~]# vi /etc/fdfs/client.conf
base_path=/tmp
tracker_server=CentOSA:22122
tracker_server=CentOSB:22122
tracker_server=CentOSC:22122
启动服务器
[root@CentOSX ~]# /etc/init.d/fdfs_trackerd start
Starting FastDFS tracker server:
[root@CentOSX ~]# /etc/init.d/fdfs_storaged start
Starting FastDFS storage server:
[root@CentOSX ~]# ps -aux | grep fdfs
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
root 78950 0.0 0.1 144784 2040 ? Sl 21:06 0:00 /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
root 79000 13.0 3.2 83520 67144 ? Sl 21:06 0:06 /usr/bin/fdfs_storaged /etc/fdfs/storage.conf
root 79324 0.0 0.0 103320 884 pts/0 S+ 21:07 0:00 grep fdfs
[root@CentOSX ~]#
FastDFS操作
上传⽂件
[root@CentOSX ~]# fdfs_upload_file /etc/fdfs/client.conf install.log
group1/M00/00/01/wKikgV0YtQ2AfmozAAAixZ60QfI792.log
下载
[root@CentOSX ~]# fdfs_download_file /etc/fdfs/client.conf
group1/M00/00/01/wKikgV0YtQ2AfmozAAAixZ60QfI792.log
查看文件信息
[root@CentOSX ~]# fdfs_file_info /etc/fdfs/client.conf
group1/M00/00/01/wKikgV0YtQ2AfmozAAAixZ60QfI792.log
source storage id: 0
source ip address: 192.168.164.129
file create timestamp: 2019-06-30 21:11:41
file size: 8901
file crc32: 2662613490 (0x9EB441F2)
删除⽂件
[root@CentOSX ~]# fdfs_delete_file /etc/fdfs/client.conf
group1/M00/00/01/wKikgV0YtQ2AfmozAAAixZ60QfI792.log
⽂件追加
[root@CentOSX ~]# clear
[root@CentOSX ~]# echo "hello" > 1.txt
[root@CentOSX ~]# echo "word" > 2.txt
[root@CentOSX ~]# fdfs_upload_appender /etc/fdfs/client.conf /root/1.txt
group2/M00/00/00/wKikgl0qX4KEep_PAAAAAIpZx2Y751.txt
[root@CentOSX ~]# fdfs_append_file /etc/fdfs/client.conf
group2/M00/00/00/wKikgl0qX4KEep_PAAAAAIpZx2Y751.txt 2.txt
[root@CentOSX ~]# fdfs_download_file /etc/fdfs/client.conf
group2/M00/00/00/wKikgl0qX4KEep_PAAAAAIpZx2Y751.txt
[root@CentOSX ~]# cat wKikgl0qX4KEep_PAAAAAIpZx2Y751.txt
hello
word
监视状态
[root@CentOSX ~]# fdfs_monitor /etc/fdfs/client.conf
⽂件校验和
[root@CentOSX ~]# fdfs_crc32 /etc/fdfs/client.conf
group2/M00/00/00/wKikgl0qX4KEep_PAAAAAIpZx2Y751.txt
2911662598
Nginx集成FastDFS
FastDFS Nginx Module等价一个client，提供访问接口，用户可以访问nginx，nginx访问所有集群中的不同storge-server节点。
web 访问nginx，nginx 访问 tracker，tacker 去找文件
Nginx配置安装
下载fastdfs-nginx-module（不建议使⽤github上，因为编译有问题）和 nginx-1.11.1.tar.gz
[root@CentOSX ~]# tar -zxf fastdfs-nginx-module.tar.gz
[root@CentOSX ~]# yum install -y pcre-devel
[root@CentOSX ~]# yum install -y openssl-devel
[root@CentOSX ~]# wget http://nginx.org/download/nginx-1.11.1.tar.gz
[root@CentOSX ~]# tar -zxf nginx-1.11.1.tar.gz
[root@CentOSX nginx-1.11.1]# ./configure --prefix=/usr/local/nginx-1.11.1/ --addmodule=/root/fastdfs-nginx-module/src
[root@CentOSX nginx-1.11.1]# make
[root@CentOSX nginx-1.11.1]# make install
拷⻉配置
[root@CentOSX ~]# cp /root/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/
[root@CentOSX ~]# cd /root/fastdfs-5.11/conf/
[root@CentOSX conf]# cp http.conf mime.types anti-steal.jpg /etc/fdfs/
配置nginx.conf
[root@CentOSA ~]# vi /usr/local/nginx-1.11.1/conf/nginx.conf
在nginx.conf中的 location 配置项前加入如下配置
location ~ /group[0-9]+/M00 {
root /data/fdfs/storage/store01;
ngx_fastdfs_module;
}
location ~ /group[0-9]+/M01 {
root /data/fdfs/storage/store02;
ngx_fastdfs_module;
}
修改mod_fastdfs.conf
[root@CentOSX ~]# vi /etc/fdfs/mod_fastdfs.conf
tracker_server=CentOSA:22122
tracker_server=CentOSB:22122
tracker_server=CentOSC:22122
# [1,2,3]对应不同的机器，第一台机器上group_name=group1,第二台...
group_name=group[1,2,3]
url_have_group_name = true
store_path_count=2
store_path0=/data/fdfs/storage/store01
store_path1=/data/fdfs/storage/store02
启动nginx
[root@CentOSX ~]# cd /usr/local/nginx-1.11.1/
# ./sbin/nginx -t 为测试是否具备启动条件
[root@CentOSX nginx-1.11.1]# ./sbin/nginx -t
ngx_http_fastdfs_set pid=116305
ngx_http_fastdfs_set pid=116305
nginx: the configuration file /usr/local/nginx-1.11.1//conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx-1.11.1//conf/nginx.conf test is successful
#此时才真正启动nginx服务
[root@CentOSX nginx-1.11.1]# ./sbin/nginx
测试下载
[root@CentOSA ~]# fdfs_upload_file /etc/fdfs/client.conf install.log
group2/M01/00/00/wKikgl0p_4-AIqGJAAAixZ60QfI693.log
随便访问⼀个nginx服务查看效果
http://CentOS[A|B|C]/group2/M01/00/00/wKikgl0p_4-AIqGJAAAixZ60QfI693.log?filename=install.log
⽤户在请求的时候，可以选择性添加⽂件名，⽤于修改下载的⽂件名
FastDHT⽂件去重
FastDFS除了提供了与nginx的集成，已提供了去重的⽂件解决⽅案。该解决⽅案FastDFS的作者yuqing也
在github上以FastDHT分⽀贡献出来了。
安装
1.安装BerkeleyDB 下载db-4.7.25.tar.gz
[root@CentOSX ~]# tar -zxf db-4.7.25.tar.gz
[root@CentOSX ~]# cd db-4.7.25
[root@CentOSX db-4.7.25]# cd build_unix/
[root@CentOSX build_unix]# ./../dist/configure
[root@CentOSX build_unix]# make
[root@CentOSX build_unix]# make install
2.安装FastDHT
[root@CentOSX ~]# tar zxf FastDHT_v2.01.tar.gz
[root@CentOSX ~]# cd FastDHT
[root@CentOSX FastDHT]# ./make.sh
[root@CentOSX FastDHT]# ./make.sh install
安装结束后会在/etc⽬录下产⽣fdht⽂件夹
[root@CentOSX FastDHT]# tree /etc/fdht/
/etc/fdht/
!"" fdht_client.conf
!"" fdhtd.conf
#"" fdht_servers.conf
3.修改fdhtd.conf
[root@CentOSX ~]# mkdir /data/fastdht
[root@CentOSX ~]# vi /etc/fdht/fdhtd.conf
base_path=/data/fastdht
4.修改fdht_servers.conf
[root@CentOSX ~]# vi /etc/fdht/fdht_servers.conf
group_count = 3
group0 = CentOSA:11411
group1 = CentOSB:11411
group2 = CentOSC:11411
5.修改fdht_client.conf配置⽂件
[root@CentOSX ~]# vi /etc/fdht/fdht_client.conf
base_path=/tmp/
6.启动FDHT服务
[root@CentOSX ~]# fdhtd /etc/fdht/fdhtd.conf start|stop|restart
[root@CentOSX ~]# ps -axu| grep fd
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
root 29127 0.0 0.8 195200 17504 ? Sl 00:38 0:00 fdhtd
/etc/fdht/fdhtd.conf start
root 29193 0.0 0.0 103320 884 pts/0 S+ 00:39 0:00 grep fd
root 128672 0.2 0.1 275856 2204 ? Sl 00:14 0:03
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
root 128710 1.8 3.2 85584 67188 ? Sl 00:14 0:27
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf
操作FastDHT服务 (测试是否安装成功)
设置值
[root@CentOSX ~]# fdht_set /etc/fdht/fdht_client.conf jiangzz:user001
name='jiangzz',age=18;
This is FastDHT client test program v2.01
Copyright (C) 2008, Happy Fish / YuQing
FastDHT may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDHT source kit.
Please visit the FastDHT Home Page http://www.csource.org/
for more detail.
success set key count: 2, fail count: 0
读取值
[root@CentOSX ~]# fdht_get /etc/fdht/fdht_client.conf jiangzz:user001 name,age
This is FastDHT client test program v2.01
Copyright (C) 2008, Happy Fish / YuQing
FastDHT may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDHT source kit.
Please visit the FastDHT Home Page http://www.csource.org/
for more detail.
name=jiangzz
age=18
success get key count: 2, fail count: 0
删除值
[root@CentOSX ~]# fdht_delete /etc/fdht/fdht_client.conf jiangzz:user001 name;
This is FastDHT client test program v2.01
Copyright (C) 2008, Happy Fish / YuQing
FastDHT may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDHT source kit.
Please visit the FastDHT Home Page http://www.csource.org/
for more detail.
success delete keys: name
success delete key count: 1, fail count: 0
集成FastDHT
1.修改etc/fdfs/storage.conf配置⽂件
[root@CentOSX ~]# vi /etc/fdfs/storage.conf
check_file_duplicate=1
keep_alive=1
#include /etc/fdht/fdht_servers.conf
2.分别启动fdhtd服务、fastfs
[root@CentOSX usr]# /usr/local/bin/fdhtd /etc/fdht/fdhtd.conf restart
[root@CentOSX usr]# /etc/init.d/fdfs_trackerd restart
[root@CentOSX usr]# /etc/init.d/fdfs_storaged restart
上产⽂件测试
[root@CentOSA ~]# fdfs_upload_file /etc/fdfs/client.conf install.log
group2/M00/00/00/wKikgl0qC4KAErBTAAAixXWAIyY133.log
[root@CentOSA ~]# fdfs_upload_file /etc/fdfs/client.conf install.log
group2/M00/00/00/wKikgl0qDAqAa0XwAAAixWB5m1c851.log
[root@CentOSB ~]# ls -l /data/fdfs/storage/store01/data/00/00/
total 20
lrwxrwxrwx. 1 root root 72 Jul 14 00:49 wKikgl0qC4KAErBTAAAixXWAIyY133.log ->
/data/fdfs/storage/store01/data/00/00/wKikgl0qC4KARrYBAAAixZ60QfI755.log
-rw-r--r--. 1 root root 8901 Jul 14 00:49 wKikgl0qC4KARrYBAAAixZ60QfI755.log
lrwxrwxrwx. 1 root root 72 Jul 14 00:51 wKikgl0qDAqAa0XwAAAixWB5m1c851.log ->
/data/fdfs/storage/store01/data/00/00/wKikgl0qC4KARrYBAAAixZ60QfI755.log
可以看出系统产⽣了wKikgl0qC4KAErBTAAAixXWAIyY133.log的两个链接
SpringBoot集成FastDFS
引⼊依赖
<dependency>
<groupId>com.github.tobato</groupId>
<artifactId>fastdfs-client</artifactId>
<version>1.26.6</version>
</dependency>
配置application.properties
fdfs.tracker-list=CentOSA:22122,CentOSB:22122,CentOSC:22122
# 配置默认缩略图
fdfs.thumb-image.height=80
fdfs.thumb-image.width=80
测试⽂件上传
@Autowired
private FastFileStorageClient fastFileStorageClient;
⽂件上传
FileInputStream inputStream = new FileInputStream("G:/素材资料/61850.png");
FastImageFile fastImageFile=new
FastImageFile(inputStream,inputStream.available(),"png",new HashSet<MetaData>());
StorePath storePath = fastFileStorageClient.uploadImage(fastImageFile);
System.out.println(storePath.getFullPath());
图⽚上传（缩略图）
FileInputStream inputStream = new FileInputStream("G:/素材资料/61850.png");
FastImageFile fastImageFile=new
FastImageFile(inputStream,inputStream.available(),"png",new HashSet<MetaData>(),new
ThumbImage(150,150));
StorePath storePath = fastFileStorageClient.uploadImage(fastImageFile);
System.out.println(storePath.getFullPath());
删除⽂件
fastFileStorageClient.deleteFile("group3","M00/00/00/wKjvgl0prl6AX6ygAL26hh1kYdE312_80
x80.png");
⽂件下载(⽤不到)
ByteArrayOutputStream baos = fastFileStorageClient.downloadFile("group3",
"M00/00/00/wKjvgl0prTSAMjeTAL26hhzQmiQ959.png", new
DownloadCallback<ByteArrayOutputStream>() {
@Override
public ByteArrayOutputStream recv(InputStream ins) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
IOUtils.copy(ins, baos);
return baos;
}
});
IOUtils.copy(new ByteArrayInputStream(baos.toByteArray()),new FileOutputStream("G:/素材
资料/baby.png"));
le(“group3”,“M00/00/00/wKjvgl0prl6AX6ygAL26hh1kYdE312_80
x80.png”);
⽂件下载(⽤不到)
```java
ByteArrayOutputStream baos = fastFileStorageClient.downloadFile("group3",
"M00/00/00/wKjvgl0prTSAMjeTAL26hhzQmiQ959.png", new
DownloadCallback<ByteArrayOutputStream>() {
@Override
public ByteArrayOutputStream recv(InputStream ins) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
IOUtils.copy(ins, baos);
return baos;
}
});
IOUtils.copy(new ByteArrayInputStream(baos.toByteArray()),new FileOutputStream("G:/素材
资料/baby.png"));
173
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
