SpringBoot整合FastDFS实现文件的上传与下载（详解+扩展）

最新推荐文章于 2024-06-18 09:05:28 发布

Keson Z

最新推荐文章于 2024-06-18 09:05:28 发布

阅读量4k

点赞数 3

分类专栏： # fastDFS # SpringBoot 文章标签： fastdfs java

本文链接：https://blog.csdn.net/weixin_44009447/article/details/108202818

版权

SpringBoot 同时被 2 个专栏收录

9 篇文章 0 订阅

订阅专栏

fastDFS

4 篇文章 0 订阅

订阅专栏

前言：FastDFS是什么呢？

FastDFS是一个开源的轻量级分布式文件系统，它对文件进行管理，功能包括：文件存储、文件同步、文件访问（文件上传、文件下载）等，解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务，如相册网站、视频网站等等。

FastDFS为互联网量身定制，充分考虑了冗余备份、负载均衡、线性扩容等机制，并注重高可用、高性能等指标，使用FastDFS很容易搭建一套高性能的文件服务器集群提供文件上传、下载等服务。

文章目录

一。基础配置

1.在pom.xml文件中导入依赖：

   <!-- fastdfs -->
  <dependency>
      <groupId>org.csource</groupId>
      <artifactId>fastdfs-client-java</artifactId>
      <version>1.29-SNAPSHOT</version>
  </dependency>

注意：该依赖在maven中心仓库中并不存在，需要手动打包并添加到类库中，具体操作方法参照我的另外一篇博客：解决Dependency ‘org.csource:fastdfs-client-java:1.29-SNAPSHOT‘ not found（解决maven仓库中找不到依赖）的问题

2.FastDFS配置文件fdfs_client.conf，其中主要是对连接FastDFS进行一些配置：

#连接超时时间
connect_timeout = 60
#网络超时时间
network_timeout = 60
#字符编码
charset = UTF-8
#端口
http.tracker_http_port = 7099
http.anti_steal_token = no
#ip地址
tracker_server = 192.168.23.222:22122

3.连接FastDFS工具类FastDFSClient ，其中包含了读取FastDFS的配置文件，文件上传与下载等方法：

import org.csource.common.MyException;
import org.csource.common.NameValuePair;
import org.csource.fastdfs.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import java.io.IOException;

public class FastDFSClient {


/**
     * it is to define an logger to record the operation's log
     */

    private static Logger logger = LoggerFactory.getLogger(FastDFSClient.class);

    static {
        try {
//            线上
//            String filePath = "/home/data/config/fdfs_client.conf";
//            线下
            String filePath = "D:\\data\\config\\fdfs_client.conf";
            ClientGlobal.init(filePath);
//            String filePath = new ClassPathResource("fdfs_client.pr operties").getFile().getAbsolutePath();
//            ClientGlobal.initByProperties(filePath);
        } catch (IOException e) {
            logger.error("Get FastDFS config file fail!",e);
        } catch (MyException e) {
            logger.error("FastDF init fail!",e);
        }
    }

    /**
     * 根据文件名，文件类型以及字节流数组上传文件
     * @param fileName
     * @param fileExt
     * @param fileBuff
     * @return
     */
    public static String[] upload(String fileName,String fileExt,byte[] fileBuff) {
        logger.info("File Name : " + fileName + ",File Length : " + fileBuff.length);
        NameValuePair[] meta_list = new NameValuePair[1];
        meta_list[0] = new NameValuePair("author","root");

        long startTime = System.currentTimeMillis();
        String[] uploadResults = null;
        StorageClient storageClient = null;

        try {
            storageClient = getTrackerClient();
            uploadResults = storageClient.upload_file(fileBuff,fileExt,meta_list);
        } catch (IOException e) {
            logger.error("IO Exception when uploading the file : " + fileName,e);
        } catch (MyException e) {
            logger.error("Non IO Exception when uploading the file : " + fileName,e);
        }
        logger.info("upload_file time used : " + (System.currentTimeMillis() - startTime) + "ms");

        if (uploadResults == null && storageClient != null) {
            logger.error("upload file fail, error code : " + storageClient.getErrorCode());
        } else {
            String groupName = uploadResults[0];
            String remoteFileName = uploadResults[1];
            logger.info("upload file successfully! " + "group_name : " + groupName + ", remoteFileName : " + remoteFileName );
        }
        return uploadResults;
    }


/**
     * @description this function is to download file from FastDFS and return the byte stream
     * @param groupName
     * @param remoteName
     * @return
     */

    public static byte[] downloadFile(String groupName, String remoteName) {
        byte[] fileByte = null;
        try {
            StorageClient storageClient = getTrackerClient();
            System.out.println("group name:"+groupName + ", remote name:" + remoteName);
            fileByte = storageClient.download_file(groupName,remoteName);
        } catch (IOException e) {
            logger.error("IO Exception: Get file from FastDFS Server failed!",e);
        } catch (MyException e) {
            logger.error("Non IO Exception: Get file from FastDFS Server failed!",e);
        }
        return fileByte;
    }


/**
     * @description this function is to delete appointed file that stored at FastDFS
     * @param groupName
     * @param remoteName
     * @return
     */

    public static Integer deleteFile(String groupName,String remoteName) {
        Integer deleteFileNums = null;
        try {
            StorageClient storageClient = getTrackerClient();
            deleteFileNums = storageClient.delete_file(groupName,remoteName);
        } catch (IOException e) {
            logger.error("IO Exception: Delete file failed!",e);
        } catch (MyException e) {
            logger.error("Non IO Exception: Delete file failed",e);
        }
        return deleteFileNums;
    }


/**
     * @description this function is to get appointed file's info from FastDFS
     * @param groupName
     * @param remoteName
     * @return
     */

    public static FileInfo getFileInfo(String groupName, String remoteName) {
        try {
            StorageClient storageClient = getTrackerClient();
            return storageClient.get_file_info(groupName,remoteName);
        } catch (IOException e) {
            logger.error("IO Exception: Get FileInfo from FastDFS failed!",e);
        } catch (MyException e) {
            logger.error("Non IO Exception: Get FileInfo from FastDFS failed!",e);
        }
        return null;
    }


/**
     * @param groupName
     * @return
     * @throws IOException
     */

    public static StorageServer[] getStoreStorages(String groupName) throws IOException, MyException {
        TrackerClient trackerClient = new TrackerClient();
        TrackerServer trackerServer = trackerClient.getConnection();
        return trackerClient.getStoreStorages(trackerServer,groupName);
    }


/**
     * @param groupName
     * @param remoteFileName
     * @return
     * @throws IOException
     */

    public static ServerInfo[] getFetchStorages(String groupName,String remoteFileName) throws IOException, MyException {
        TrackerClient trackerClient = new TrackerClient();
        TrackerServer trackerServer = trackerClient.getConnection();
        return trackerClient.getFetchStorages(trackerServer,groupName,remoteFileName);
    }

    public static String getFilePath(String groupName, String remoteFileName) throws IOException {
//        线上
//        String filePath = "http://218.192.110.129:7099/" + groupName + "/" +remoteFileName;
//        本地
        String filePath = getTrackerUrl() + groupName + "/" +remoteFileName;
        System.out.println("filePath: " + filePath);
        return filePath;
    }


/**
     * @return
     * @throws IOException
     */

    private static String getTrackerUrl() throws IOException {
        return "http://" + getTrackerServer().getInetSocketAddress().getHostString() + ":" + ClientGlobal.getG_tracker_http_port() + "/";
    }



/**
     * @description this function is to get the FastDFS's storageClient
     * @return
     * @throws IOException
     */

    private static StorageClient getTrackerClient() throws IOException {
        TrackerServer trackerServer = getTrackerServer();
        StorageClient storageClient = new StorageClient(trackerServer,null);
        return storageClient;
    }


/**
     * @description this function is to get the FastDFS's trackerClient
     * @return
     * @throws IOException
     */

    private static TrackerServer getTrackerServer() throws IOException {
        TrackerClient trackerClient = new TrackerClient();
        TrackerServer trackerServer = trackerClient.getConnection();
        return trackerServer;
    }
}

二。文件上传

1.文件上传的主要实现思路是将上传文件转换成byte数组，获取文件上传名称以及文件类型，调用StorageClient类中的upload_file方法，该方法执行成功后返回上传的所属组别以及远程文件名称，再用/连接得到访问url，控制器上传接口代码如下：

/**
     * 发送个人站内信
     *
     * @param jwt
     * @param toIds
     * @param title
     * @param content
     * @param multipartFile
     * @return
     */
@ApiOperation(value = "发送个人站内信")
    @PostMapping("/send")
    public HttpRespond send(
            @RequestHeader("Authorization") String jwt,
            @RequestParam(value = "toIds") Integer[] toIds,
            @RequestParam(value = "title") String title,
            @RequestParam(value = "content") String content,
            @RequestParam(value = "multipartFile") MultipartFile multipartFile) {

        // 获取当前登录的用户信息: userId,
        HashMap<String, String> userInfo = JsonInfoUtils.getUserInfo(jwt);
        if (userInfo == null) {
            return HttpRespond.unauthorized();
        }
        Integer userId = Integer.valueOf(userInfo.get("userId"));

        // 接收人不能为空
        if (toIds == null || toIds.length <= 0) {
            return HttpRespond.badRequest();
        }

        // 开启事务管理
        DefaultTransactionDefinition def = new DefaultTransactionDefinition();
        def.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
        TransactionStatus status = txManager.getTransaction(def);
        String tip="";
        try {
            // 1. 插入 "站内信" 数据
            Message message = new Message();
            message.setTitle(title);
            message.setContent(content);
            message.setType(MessageTypeEnum.NORMAL_MAIL.getCode());
            messageService.save(message);

            // 2，插入 "站内信用户关联" 数据
            MessageToUser messageToUser = new MessageToUser();
            messageToUser.setMessageId(message.getId());
            messageToUser.setReadStatus(false);
            messageToUser.setSendId(userId);
            for (Integer toId : toIds) {
                messageToUser.setRecId(toId);
                // 插入数据
                messageToUserService.save(messageToUser);

                // 发送websocket提示
                messageWsMsgHandler.sendUnreadCount(toId);
            }

            //3.上传附件
            tip=saveAttachment(multipartFile,message.getId(),MessageAttachmentTargetTypeEnum.MESSAGE.getCode());
        } catch (Exception e) {
            txManager.rollback(status);
            log.error("系统内部错误", e);
            return HttpRespond.unknownException();
        }
        txManager.commit(status);

        //如果上传附件类型有误
        if(tip.equals("wrongFileType")){
            return HttpRespond.wrongFileType();
        }
        //如果上传文件大小超过100M
        if(tip.equals("wrongFileSize")){
            return HttpRespond.wrongFileSize();
        }

        return HttpRespond.success();
    }



/**
     * 上传保存邮件附件
     * 默认支持附件上传类型（jpg,png,txt,doc,pdf,zip,rar）
     * 默认上传文件大小（100m）
     */
    private String saveAttachment(MultipartFile multipartFile, int targetId,int targetType) {
        // 附件上传并插入 "附件" 数据
        //获取上传的原始文件全称
        String fileOriName = multipartFile.getOriginalFilename();
        //获取上传的文件名称
        String fileName = fileOriName.substring(0, fileOriName.lastIndexOf("."));
        String type="jpg,png,txt,doc,pdf,zip,rar";
        //获取上传文件的后缀名（文件类型）
        String fileExt = fileOriName.substring(fileOriName.lastIndexOf(".") + 1);
        //上传文件格式不符合
        if(!type.contains(fileExt)){
            return "wrongFileType";
        }

        long length = multipartFile.getSize();
        //获取上传文件大小
        double fileSize = FileSizeUtil.getFileSize(length, "B");
        //如果上传文件大小大于100m
        if (fileSize/1048576>100){
            return "wrongFileSize";
        }

        String size = fileSize + "B";
        String[] uploadResults = null;
        byte[] fileBuff = null;
        InputStream inputStream = null;
        try {
            //接受字符文件输入流
            inputStream = multipartFile.getInputStream();
            if (inputStream != null) {
                //获取数据流中含有字节数
                int len1 = inputStream.available();
                fileBuff = new byte[len1];
                //从(来源)输入流中(读取内容)读取的一定数量字节数,并将它们存储到(去处)缓冲区数组fileBuff中
                inputStream.read(fileBuff);
            }
        } catch (IOException e) {
            log.error("IO Exception : add file exception!", e);
        } finally {
            try {
                inputStream.close();
            } catch (IOException e) {
                log.error("IO Exception : close Input Stream error!", e);
            }
        }
        //上传文件，返回所属组别以及远程文件名称
        uploadResults = FastDFSClient.upload(fileName, fileExt, fileBuff);
        String groupName = uploadResults[0];
        String remoteName = uploadResults[1];
        String url = null;
        try {
            //得到上传后远程文件的访问路径
            url = FastDFSClient.getFilePath(groupName, remoteName);
        } catch (IOException e) {
            log.error("IO Exception : close Input Stream error!", e);
        }

        //封装数据
        MessageAttachment attachment = new MessageAttachment();
        attachment.setTargetId(targetId);
        attachment.setTargetType(targetType);
        attachment.setOriginalName(fileOriName);
        attachment.setActualName(fileName);
        attachment.setSize(size);
        attachment.setUrl(url);
        attachment.setFileType(fileExt);

        attachmentService.save(attachment);

        return "success";
    }

2.返回上传文件大小工具类：

/**
 * @Description：返回上传文件大小
 * @Author :zks
 * @Date :9:55 2020/8/20
 */
public class FileSizeUtil {

    /**
     * length:文件长度（multipartFile.getSize())
     * unit限制单位（B,K,M,G）
     * @param length
     * @param unit
     * @return
     */
    public static Double getFileSize(long length,String unit){
        double fileSize = 0;
        if ("B".equals(unit.toUpperCase())) {
            fileSize = (double) length;
        } else if ("K".equals(unit.toUpperCase())) {
            fileSize = (double) length / 1024;
        } else if ("M".equals(unit.toUpperCase())) {
            fileSize = (double) length / 1048576;
        } else if ("G".equals(unit.toUpperCase())) {
            fileSize = (double) length / 1073741824;
        }
        return fileSize;
    }
}

3.使用postman进行测试：

在这里插入图片描述

4.数据库表结构如下：

在这里插入图片描述

5.插入的数据如下：

16
19
1
1575635185311.jpg
1575635185311
84924.0B
http://192.168.23.222:7099/group1/M00/00/02/wKgX3l9DjXmAUiFeAAFLvP-tZBQ642.jpg
jpg
2020-08-24 17:50:49
2020-08-24 17:50:49 0

6.根据数据库表信息查看上传文件，路径为：http://192.168.23.222:7099/group1/M00/00/02/wKgX3l9DjXmAUiFeAAFLvP-tZBQ642.jpg

在这里插入图片描述

注意：该访问地址由ip+端口+上传组别+fastdfs内部定义上传位置+内部定义上传名称

三。文件下载

1.文件下载主要实现思路是从数据库拿到访问该文件的远程url，再以/将其进行切割，得到该文件所在组名以及文件地址，调用StorageClient类中的download_file方法，该方法执行成功返回该文件的字节数组。先根据接口传进来的下载位置参数，以及数据库表存储的文件名称生成新文件，再调用FileUtils类中的writeByteArrayToFile方法将返回的byte数组写入到新文件中，控制器下载接口代码如下：

/**
     * 附件下载
     *
     * @param id
     * @param location
     * @return
     * @throws IOException
     */
    @ApiOperation(value = "根据附件id下载附件")
    @GetMapping("/download")
    public HttpRespond download(@RequestParam("id") Integer id, @RequestParam("location") String location) throws IOException {
        //根据id查找附件信息
        MessageAttachment messageAttachment = attachmentService.getById(id);

        //  http://192.168.23.222:7099/group1/M00/00/02/wKgX3l8-F9KABdUlAAP5BN0hxz4168.pdf
        String s = messageAttachment.getUrl();
        //以“/”切割4次
        String[] ss = s.split("/", 5);

        //根据fastdfs上传组别以及资源名称得到下载字节数组  ss[3]:group1  ss[4]:M00/00/02/wKgX3l8-F9KABdUlAAP5BN0hxz4168.pdf
        byte[] bytes = FastDFSClient.downloadFile(ss[3], ss[4]);
        // 给定文件路径和名称,把"\"替换成"/",target为转义字符，注意替换后重新给字符串赋值
        //D:\fastdfsDownload ----->D:/fastdfsDownload
        location = location.replace("\\", "/");
        String pathName = location + "/" + messageAttachment.getOriginalName();
        File file = new File(pathName);
        //将字节流数组写入文件
        FileUtils.writeByteArrayToFile(file, bytes);

        return HttpRespond.success(pathName);
    }

2.使用postman测试该下载接口：

在这里插入图片描述
注意：上面我用红圈标注的传入参数格式你是不是觉得很奇怪？其实那就是一个下载地址，当我们传入参数中带有+，空格，/，?，%，#，&，=等特殊符号的时候，可能在服务器端无法获得正确的参数值，此时需要对字符进行转译。这里我们对：以及\进行了转译，对应转译字符分别为%3A和%2F。