fastDFS分布式文件系统
什么是fastDFS
fastDFS是一个底层使用C语言编写并且开源的, 分布式文件系统
作用:可以统一管理存储服务器集群, 统一管理存储读取文件.
fastDFS的原理
fastDFS原理(工作流程)
- fastDFS一共分为三部分, client客户端(我们的项目), trackerServer管理端, storageServer存储端
- client客户端(我们的项目): 使用fastDFS的jar包中的api来存储文件,调用fastDFS
- trackerServer管理端:管理storageServer存储端集群的, 管理端如果死掉存储端无法使用, 所以管理端都是做双机热备, 就是一台主机, 一台备机, 他们之间做心跳检测机制, 平时主机工作, 备机向主机发送ping命令, 主机接收到后返回pong命令, 如果一段时间内没有返回, 备机会认为主机死掉会替代主机工作.这样可以保证管理端的高可用. 管理端还有负载均衡的功能, 可以平均分配请求给存储端. 这样就可以承载高并发的存储需求.
- storageServer存储端: 它负责具体存储文件, 也是两台为一组, 一台主机一台备机, 之间双机热备, 做心跳检测.保证存储端的高可用. 存储端做冗余存储, 就是我们向存储端主机存储内容, 主机会向备机发送内容,主机和备机存储的内容是一样的, 所以主机即使坏掉, 备机也有同样的内容可以使用.这种叫容灾配置.存储端理论上存储容量是无限的, 因为两台为一组机器可以无限扩展.
工作流程: 我们项目存储的时候先调用管理端, 管理端会给我们分配一台存储端的IP地址和端口, 我们向存储端机器存储文件, 存储端会给我们返回存储后的地址以及文件名, 文件名会自动被存储端重命名, 防止文件重名
优点:
- 管理端有负载均衡的功能, 可以承载高并发的存储需求
- 存储端可以无限扩展, 理论上存储容量是无限的, 扩展性好
- 存储端和管理端都使用了心跳检测机制, 保证了服务器的高可用
- 存储端做了冗余存储, 所以即使有机器坏掉也不会丢数据, 容灾性好
缺点: fastDFS结构复杂, 会使项目中的复杂度变高, 并且搭建fastDFS会使用很多服务器, 比较贵.
fastDFS的例子
一、创建Maven的普通项目
二、引入依赖:注意fastDFS的jar包在maven中央仓库中没有,需要去fastDFS的官网下载,然后执行命令
mvn install:install-file -DgroupId=org.csource.fastdfs -DartifactId=fastdfs -Dversion=1.2 -Dpackaging=jar -Dfile=d:\setup\fastdfs_client_v1.20.jar
<dependency>
<groupId>org.csource.fastdfs</groupId>
<artifactId>fastdfs</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.1</version>
</dependency>
三、添加配置文件:fdfs_client.conf
,注意文件名不能改动
# connect timeout in seconds
# default value is 30s
connect_timeout=30
# network timeout in seconds
# default value is 30s
network_timeout=60
# the base path to store log files
base_path=/home/fastdfs
# tracker_server can ocur more than once, and tracker_server format is
# "host:port", host can be hostname or ip address
tracker_server=192.168.200.128:22122
#standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level=info
# if use connection pool
# default value is false
# since V4.05
use_connection_pool = false
# connections whose the idle time exceeds this time will be closed
# unit: second
# default value is 3600
# since V4.05
connection_pool_max_idle_time = 3600
# if load FastDFS parameters from tracker server
# since V4.05
# default value is false
load_fdfs_parameters_from_tracker=false
# if use storage ID instead of IP address
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# default value is false
# since V4.05
use_storage_id = false
# specify storage ids filename, can use relative or absolute path
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# since V4.05
storage_ids_filename = storage_ids.conf
#HTTP settings
http.tracker_server_port=80
#use "#include" directive to include HTTP other settiongs
##include http.conf
四、代码实现
- 加载配置文件(绝对路径)
- 创建管理端对象
- 通过管理端对象获取连接
- 创建存储端对象
- 上传文件
package cn.itcast.test;
import org.csource.common.NameValuePair;
import org.csource.fastdfs.ClientGlobal;
import org.csource.fastdfs.StorageClient1;
import org.csource.fastdfs.TrackerClient;
import org.csource.fastdfs.TrackerServer;
public class TestFastDFS {
public static void main(String[] args) throws Exception {
//1.加载配置文件
ClientGlobal.init("D:\\Program Files\\IDEA Projects\\fastDFSDemo\\src\\main\\resources\\fdfs_client.conf");
//2.创建管理端对象
TrackerClient trackerClient = new TrackerClient();
//3.通过管理端对象获取连接
TrackerServer connection = trackerClient.getConnection();
//4.创建存储端对象
StorageClient1 storageClient1 = new StorageClient1(connection,null);
//创建文件属性信息对象数组
NameValuePair[] meta_list = new NameValuePair[3];
meta_list[0] = new NameValuePair("fileName","1");
meta_list[1] = new NameValuePair("ExtName","png");
meta_list[2] = new NameValuePair("author","sm");
//5.上传文件
String path = storageClient1.upload_file1("D:\\1.png", "png", meta_list);
System.out.println(path);// group1/M00/00/01/wKjIgF7cceeAC36TAAAftfpwV-E664.png
}
}
五、访问图片
http://192.168.200.128/group1/M00/00/01/wKjIgF7cceeAC36TAAAftfpwV-E664.png
fastDFS工具类
package cn.itcast.core.util;
import org.apache.commons.io.FilenameUtils;
import org.csource.common.NameValuePair;
import org.csource.fastdfs.ClientGlobal;
import org.csource.fastdfs.StorageClient1;
import org.csource.fastdfs.StorageServer;
import org.csource.fastdfs.TrackerClient;
import org.csource.fastdfs.TrackerServer;
public class FastDFSClient {
private TrackerClient trackerClient = null;
private TrackerServer trackerServer = null;
private StorageServer storageServer = null;
private StorageClient1 storageClient = null;
public FastDFSClient(String conf) throws Exception {
if (conf.contains("classpath:")) {
conf = conf.replace("classpath:", this.getClass().getResource("/").getPath().replace("%20"," "));
}
ClientGlobal.init(conf);
trackerClient = new TrackerClient();
trackerServer = trackerClient.getConnection();
storageServer = null;
storageClient = new StorageClient1(trackerServer, storageServer);
}
/**
* 上传文件方法
* <p>Title: uploadFile</p>
* <p>Description: </p>
* @param fileName 文件全路径
* @param extName 文件扩展名,不包含(.)
* @param metas 文件扩展信息
* @return
* @throws Exception
*/
public String uploadFile(String fileName, String extName, NameValuePair[] metas) throws Exception {
String result = storageClient.upload_file1(fileName, extName, metas);
return result;
}
/**
*这个比较好用
*/
public String uploadFile(byte[] file, String fileName, long fileSize) throws Exception {
NameValuePair[] metas = new NameValuePair[3];
metas[0] = new NameValuePair("fileName", fileName);
metas[1] = new NameValuePair("fileSize", String.valueOf(fileSize));
metas[2] = new NameValuePair("fileExt", FilenameUtils.getExtension(fileName));
String result = storageClient.upload_file1(file, FilenameUtils.getExtension(fileName), metas);
return result;
}
public String uploadFile(String fileName) throws Exception {
return uploadFile(fileName, null, null);
}
public String uploadFile(String fileName, String extName) throws Exception {
return uploadFile(fileName, extName, null);
}
/**
* 上传文件方法
* <p>Title: uploadFile</p>
* <p>Description: </p>
* @param fileContent 文件的内容,字节数组
* @param extName 文件扩展名
* @param metas 文件扩展信息
* @return
* @throws Exception
*/
public String uploadFile(byte[] fileContent, String extName, NameValuePair[] metas) throws Exception {
String result = storageClient.upload_file1(fileContent, extName, metas);
return result;
}
public String uploadFile(byte[] fileContent) throws Exception {
return uploadFile(fileContent, null, null);
}
public String uploadFile(byte[] fileContent, String extName) throws Exception {
return uploadFile(fileContent, extName, null);
}
}
文件上传
/**
* 文件上传
*/
@RestController
@RequestMapping("/upload")
public class UploadController {
/**
* 读取application.properties配置文件内容,通过里面的key,FILE_SERVER_URL,读取值并赋值给FILE_SERVER
*/
@Value("${FILE_SERVER_URL}")
private String FILE_SERVER;
@RequestMapping("/uploadFile")
public Result uploadFile(MultipartFile file) throws Exception {
try {
FastDFSClient fastDFS = new FastDFSClient("classpath:fastDFS/fdfs_client.conf");
//上传文件,返回文件保存的路径和文件名
String path = fastDFS.uploadFile(file.getBytes(), file.getOriginalFilename(), file.getSize());
return new Result(true,FILE_SERVER+path);
} catch (Exception e) {
e.printStackTrace();
return new Result(false,"上传失败!");
}
}
}
补充:elementui与vue的文件上传
<template>
<div>
<el-upload
class="upload-demo"
action="http://localhost:8080/upload/uploadFile"
:on-preview="handlePreview"
:on-remove="handleRemove"
:before-remove="beforeRemove"
multiple
:limit="1"
:on-exceed="handleExceed"
:file-list="fileList">
<el-button size="small" type="primary">点击上传</el-button>
</el-upload>
</div>
</template>
<script>
export default {
data() {
return {
fileList: []
};
},
methods: {
handleRemove(file, fileList) {
console.log(file, fileList);
},
handlePreview(file) {
console.log(file);
},
handleExceed(files, fileList) {
this.$message.warning(`当前限制选择 3 个文件,本次选择了 ${files.length} 个文件,共选择了 ${files.length + fileList.length} 个文件`);
},
beforeRemove(file, fileList) {
return this.$confirm(`确定移除 ${ file.name }?`);
}
}
}
</script>
<style>
</style>
Java中json的转换
使用阿里的fastjson
一、将json格式的数组字符串转化成List集合
JSON.parseArray
将json格式字符串转换成List集合
二、将json格式字符串转换成实体对象
JSON.parseObject
将json格式字符串转换成pojo或map对象