## 搭建fastDFS分布式文件系统
-
安装gcc环境
yum -y install gcc-c++
-
安装fastDFS依赖libevent库
yum -y install libevent
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-wqefY0lZ-1576842250271)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220171303780.png)]
-
上传所需文件
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-qhStyDZB-1576842250273)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220171650755.png)]
-
安装libfastcommon
- 解压
- 执行指令编译安装
1. tar -zxf libfastcommonV1.0.7.tar.gz 2. cd libfastcommon-1.0.7/ 3. ./make.sh 4. ./make.sh install
-
将/usr/lib64拷贝到/usr/lib下面
cp libfastcommon.so /usr/lib/
验证是否查看成功:
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-zyQGaDBs-1576842250273)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220172924367.png)]
-
安装FastDFS
-
解压fastDFS
tar -zxf FastDFS_v5.05.tar.gz
-
进入目录
cd FastDFS/
-
编译并安装
./make.sh ./make.sh install
-
查看相应的配置文件,默认安装会存在以下配置文件
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2Stwe35r-1576842250275)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220174756340.png)]
-
复制其他相应文件方便后面配合nginx的使用
cd /usr/local/fastdfs/FastDFS/ cd conf cp * /etc/fdfs/
-
安装Track服务
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-YHonyJxG-1576842250276)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220175503431.png)]
修改下述文件
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-aT86MKOJ-1576842250276)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220175620292.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-E9tOiRCa-1576842250277)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220175933422.png)]
-
启动tracker服务
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
相关指令:[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-nRb46u2V-1576842250278)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220180435555.png)]
-
安装storage服务
修改相应的配置文件(storage.cof):
# is this config file disabled # false for enabled # true for disabled disabled=false # the name of the group this storage server belongs to # # comment or remove this item for fetching from tracker server, # in this case, use_storage_id must set to true in tracker.conf, # and storage_ids.conf must be configed correctly. group_name=group1 # bind an address of this host # empty for bind all addresses of this host bind_addr= # if bind an address of this host when connect to other servers # (this storage server as a client) # true for binding the address configed by above parameter: "bind_addr" # false for binding any address of this host client_bind=true # the storage server port port=23000 # connect timeout in seconds # default value is 30s connect_timeout=30 # network timeout in seconds # default value is 30s network_timeout=60 # heart beat interval in seconds heart_beat_interval=30 # disk usage report interval in seconds stat_report_interval=60 # the base path to store data and log files base_path=/usr/local/fastdfs/FastDFS/storage # max concurrent connections the server supported # default value is 256 # more max_connections means more memory will be used max_connections=256 # the buff size to recv / send data # this parameter must more than 8KB # default value is 64KB # since V2.00 buff_size = 256KB # accept thread count # default value is 1 # since V4.07 accept_threads=1 # work thread count, should <= max_connections # work thread deal network io # default value is 4 # since V2.00 work_threads=4 # if disk read / write separated ## false for mixed read and write ## true for separated read and write # default value is true # since V2.00 disk_rw_separated = true # disk reader thread count per store base path # for mixed read / write, this parameter can be 0 # default value is 1 # since V2.00 disk_reader_threads = 1 # disk writer thread count per store base path # for mixed read / write, this parameter can be 0 # default value is 1 # since V2.00 disk_writer_threads = 1 # when no entry to sync, try read binlog again after X milliseconds # must > 0, default value is 200ms sync_wait_msec=50 # after sync a file, usleep milliseconds # 0 for sync successively (never call usleep) sync_interval=0 # storage sync start time of a day, time format: Hour:Minute # Hour from 0 to 23, Minute from 0 to 59 sync_start_time=00:00 # storage sync end time of a day, time format: Hour:Minute # Hour from 0 to 23, Minute from 0 to 59 sync_end_time=23:59 # write to the mark file after sync N files # default value is 500 write_mark_file_freq=500 # path(disk or mount point) count, default value is 1 store_path_count=1 # store_path#, based 0, if store_path0 not exists, it's value is base_path # the paths must be exist store_path0=/usr/local/fastdfs/FastDFS/storage #store_path1=/home/yuqing/fastdfs2 # subdir_count * subdir_count directories will be auto created under each # store_path (disk), value can be 1 to 256, default value is 256 subdir_count_per_path=256 # tracker_server can ocur more than once, and tracker_server format is # "host:port", host can be hostname or ip address tracker_server=192.168.0.125:22122 #standard log level as syslog, case insensitive, value list: ### emerg for emergency ### alert ### crit for critical ### error ### warn for warning ### notice ### info ### debug log_level=info #unix group name to run this program, #not set (empty) means run by the group of current user run_by_group= #unix username to run this program, #not set (empty) means run by current user run_by_user= # allow_hosts can ocur more than once, host can be hostname or ip address, # "*" means match all ip addresses, can use range like this: 10.0.1.[1-15,20] or # host[01-08,20-25].domain.com, for example: # allow_hosts=10.0.1.[1-15,20] # allow_hosts=host[01-08,20-25].domain.com allow_hosts=* # the mode of the files distributed to the data path # 0: round robin(default) # 1: random, distributted by hash code file_distribute_path_mode=0 # valid when file_distribute_to_path is set to 0 (round robin), # when the written file count reaches this number, then rotate to next path # default value is 100 file_distribute_rotate_count=100 # call fsync to disk when write big file # 0: never call fsync # other: call fsync when written bytes >= this bytes # default value is 0 (never call fsync) fsync_after_written_bytes=0 # sync log buff to disk every interval seconds # must > 0, default value is 10 seconds sync_log_buff_interval=10 # sync binlog buff / cache to disk every interval seconds # default value is 60 seconds sync_binlog_buff_interval=10 # sync storage stat info to disk every interval seconds # default value is 300 seconds sync_stat_file_interval=300 # thread stack size, should >= 512KB # default value is 512KB thread_stack_size=512KB # the priority as a source server for uploading file. # the lower this value, the higher its uploading priority. # default value is 10 upload_priority=10 # the NIC alias prefix, such as eth in Linux, you can see it by ifconfig -a # multi aliases split by comma. empty value means auto set by OS type # default values is empty if_alias_prefix= # if check file duplicate, when set to true, use FastDHT to store file indexes # 1 or yes: need check # 0 or no: do not check # default value is 0 check_file_duplicate=0 # file signature method for check file duplicate ## hash: four 32 bits hash code ## md5: MD5 signature # default value is hash # since V4.01 file_signature_method=hash # namespace for storing file indexes (key-value pairs) # this item must be set when check_file_duplicate is true / on key_namespace=FastDFS # set keep_alive to 1 to enable persistent connection with FastDHT servers # default value is 0 (short connection) keep_alive=0 # you can use "#include filename" (not include double quotes) directive to # load FastDHT server list, when the filename is a relative path such as # pure filename, the base path is the base path of current/this config file. # must set FastDHT server list when check_file_duplicate is true / on # please see INSTALL of FastDHT for detail ##include /home/yuqing/fastdht/conf/fdht_servers.conf # if log to access log # default value is false # since V4.00 use_access_log = false # if rotate the access log every day # default value is false # since V4.00 rotate_access_log = false # rotate access log time base, time format: Hour:Minute # Hour from 0 to 23, Minute from 0 to 59 # default value is 00:00 # since V4.00 access_log_rotate_time=00:00 # if rotate the error log every day # default value is false # since V4.02 rotate_error_log = false # rotate error log time base, time format: Hour:Minute # Hour from 0 to 23, Minute from 0 to 59 # default value is 00:00 # since V4.02 error_log_rotate_time=00:00 # rotate access log when the log file exceeds this size # 0 means never rotates log file by log file size # default value is 0 # since V4.02 rotate_access_log_size = 0 # rotate error log when the log file exceeds this size # 0 means never rotates log file by log file size # default value is 0 # since V4.02 rotate_error_log_size = 0 # keep days of the log files # 0 means do not delete old log files # default value is 0 log_file_keep_days = 0 # if skip the invalid record when sync file # default value is false # since V4.02 file_sync_skip_invalid_record=false # if use connection pool # default value is false # since V4.05 use_connection_pool = false # connections whose the idle time exceeds this time will be closed # unit: second # default value is 3600 # since V4.05 connection_pool_max_idle_time = 3600 # use the ip address of this storage server if domain_name is empty, # else this domain name will ocur in the url redirected by the tracker server http.domain_name= # the port of the web server on this storage server http.server_port=8888
-
启动服务
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IwizgyyA-1576842250278)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220181742453.png)]
-
配置客户端
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-CRTkrynt-1576842250279)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220181853198.png)]
-
进入客户端目录
cd /usr/local/fastdfs/FastDFS/client/
-
复制cp libfdfsclient.so 到/usr/lib中去
cp libfdfsclient.so /usr/lib /usr/local/fastdfs/FastDFS/client
-
修改client。conf文件
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-oNHzXYUJ-1576842250279)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220182314928.png)]
-
修改后文件(base_path和tracker.server)
# connect timeout in seconds # default value is 30s connect_timeout=30 # network timeout in seconds # default value is 30s network_timeout=60 # the base path to store log files base_path=/usr/local/fastdfs/FastDFS/client # tracker_server can ocur more than once, and tracker_server format is # "host:port", host can be hostname or ip address tracker_server=192.168.0.125:22122 #standard log level as syslog, case insensitive, value list: ### emerg for emergency ### alert ### crit for critical ### error ### warn for warning ### notice ### info ### debug log_level=info # if use connection pool # default value is false # since V4.05 use_connection_pool = false # connections whose the idle time exceeds this time will be closed # unit: second # default value is 3600 # since V4.05 connection_pool_max_idle_time = 3600 # if load FastDFS parameters from tracker server # since V4.05 # default value is false load_fdfs_parameters_from_tracker=false # if use storage ID instead of IP address # same as tracker.conf # valid only when load_fdfs_parameters_from_tracker is false # default value is false # since V4.05 use_storage_id = false # specify storage ids filename, can use relative or absolute path # same as tracker.conf # valid only when load_fdfs_parameters_from_tracker is false # since V4.05 storage_ids_filename = storage_ids.conf #HTTP settings http.tracker_server_port=80 #use "#include" directive to include HTTP other settiongs ##include http.conf
-
-
测试上传文件:
/usr/bin/fdfs_test /etc/fdfs/client.conf upload /home/cc.jpg http://192.168.0.125/group1/M00/00/00/wKgAfV38okmAFfwCAACkwUrTQrI399_big.jpg
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-0NfWcF4O-1576842250280)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220182917857.png)]
-
修改防火墙;
-
查看防火墙状态
firewall-cmd –state
-
停止firewall
systemctl stop firewalld.service
-
禁止firewall开机启动
systemctl disable firewalld.service
-
安装nginx和nginx的插件
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-CKaG0Blq-1576842250281)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220185135262.png)]
yum install -y pcre pcre-devel
yum install -y zlib zlib-devel
yum install -y openssl openssl-devel
-
首先解压fastdfs-nginx-module_v1.16.tar.gz
tar zxf fastdfs-nginx-module_v1.16.tar.gz
-
进入安装目录
cd fastdfs-nginx-module/
-
修改配置文件,删除其中的local
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-RVgo6c2D-1576842250281)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220191201440.png)]
-
将mod_fastdfs.conf复制到/etc/fdfs目录下,并修改。**cp mod_fastdfs.conf /etc/fdfs/ ** 主要修改存贮路径和tracker 的位置
# connect timeout in seconds # default value is 30s connect_timeout=2 # network recv and send timeout in seconds # default value is 30s network_timeout=30 # the base path to store log files base_path=/tmp # if load FastDFS parameters from tracker server # since V1.12 # default value is false load_fdfs_parameters_from_tracker=true # storage sync file max delay seconds # same as tracker.conf # valid only when load_fdfs_parameters_from_tracker is false # since V1.12 # default value is 86400 seconds (one day) storage_sync_file_max_delay = 86400 # if use storage ID instead of IP address # same as tracker.conf # valid only when load_fdfs_parameters_from_tracker is false # default value is false # since V1.13 use_storage_id = false # specify storage ids filename, can use relative or absolute path # same as tracker.conf # valid only when load_fdfs_parameters_from_tracker is false # since V1.13 storage_ids_filename = storage_ids.conf # FastDFS tracker_server can ocur more than once, and tracker_server format is # "host:port", host can be hostname or ip address # valid only when load_fdfs_parameters_from_tracker is true tracker_server=192.168.0.125:22122 # the port of the local storage server # the default value is 23000 storage_server_port=23000 # the group name of the local storage server group_name=group1 # if the url / uri including the group name # set to false when uri like /M00/00/00/xxx # set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx # default value is false url_have_group_name = true # path(disk or mount point) count, default value is 1 # must same as storage.conf store_path_count=1 # store_path#, based 0, if store_path0 not exists, it's value is base_path # the paths must be exist # must same as storage.conf store_path0=/usr/local/fastdfs/FastDFS/storage #store_path1=/home/yuqing/fastdfs1 # standard log level as syslog, case insensitive, value list: ### emerg for emergency ### alert ### crit for critical ### error ### warn for warning ### notice ### info ### debug log_level=info # set the log filename, such as /usr/local/apache2/logs/mod_fastdfs.log # empty for output to stderr (apache and nginx error_log file) log_filename= # response mode when the file not exist in the local file system ## proxy: get the content from other storage server, then send to client ## redirect: redirect to the original storage server (HTTP Header is Location) response_mode=proxy # the NIC alias prefix, such as eth in Linux, you can see it by ifconfig -a # multi aliases split by comma. empty value means auto set by OS type # this paramter used to get all ip address of the local host # default values is empty if_alias_prefix= # use "#include" directive to include HTTP config file # NOTE: #include is an include directive, do NOT remove the # before include #include http.conf # if support flv # default value is false # since v1.15 flv_support = true # flv file extension name # default value is flv # since v1.15 flv_extension = flv # set the group count # set to none zero to support multi-group # set to 0 for single group only # groups settings section as [group1], [group2], ..., [groupN] # default value is 0 # since v1.14 group_count = 0 # group settings for group #1 # since v1.14 # when support multi-group, uncomment following section #[group1] #group_name=group1 #storage_server_port=23000 #store_path_count=2 #store_path0=/home/yuqing/fastdfs #store_path1=/home/yuqing/fastdfs1 # group settings for group #2 # since v1.14 # when support multi-group, uncomment following section as neccessary #[group2] #group_name=group2 #storage_server_port=23000 #store_path_count=1 #store_path0=/home/yuqing/fastdfs
-
解压nginx
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-raldJ0vB-1576842250282)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220192514578.png)]
1.配置nginx绑定fastdfs-nginx-module ./configure --add-module=/usr/local/fastdfs/fastdfs-nginx-module/src 2.编译 make 3.安装 make install
-
配置安装之后的nginx(nginx.conf)
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-rRd1xWkl-1576842250282)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220193705227.png)]
location /group1/M00/ { ngx_fastdfs_module; }
-
进入nginx的配置目录
进入 cd sbin/ 启动nginx ./nginx 查看nginx是否启动 ps -aux | grep nginx
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-kGQ7RlNy-1576842250283)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20191220194230955.png)]
-
后续测试即可!!!
e=group2
#storage_server_port=23000
#store_path_count=1
#store_path0=/home/yuqing/fastdfs
5. 解压nginx
[外链图片转存中...(img-raldJ0vB-1576842250282)]
1.配置nginx绑定fastdfs-nginx-module
./configure --add-module=/usr/local/fastdfs/fastdfs-nginx-module/src
2.编译
make
3.安装
make install
6. 配置安装之后的nginx(nginx.conf)
[外链图片转存中...(img-rRd1xWkl-1576842250282)]
location /group1/M00/ {
ngx_fastdfs_module;
}
7. 进入nginx的配置目录
进入
cd sbin/
启动nginx
./nginx
查看nginx是否启动
ps -aux | grep nginx
[外链图片转存中...(img-kGQ7RlNy-1576842250283)]
8. 后续测试即可!!!