SpringBoot整合文件处理

文件操作

1、压缩/解压

压缩

利用Java.util.zip包中的ZipOutputStream 实现文件的压缩

ZipOutputStream (OutputStream out) 创建新的zip输出流
void putNextEntry(ZipEntry e) 开始写入新的zip文件条目并将流定位到条目数据的开始处
条目指的是一个文件夹下的多个文件。
ZipEntry(String name) 使用指定名称创建新的zip条目
ZipIutputStream实现文件的解压
ZipIutputStream (IutputStream out) 创建新的zip输入流
ZipEntry getNextEntry（）读取下一个zip条目并将流定位到该条目数据的开始处

实战

/**
 * 压缩
 */
public void zip(String input, String output, String name) throws Exception {
    //要生成的压缩文件
    ZipOutputStream out = new ZipOutputStream(new FileOutputStream(output));
    //支持多个文件压缩在一起
    String[] paths = input.split("\\|");
    File[] files = new File[paths.length];
    byte[] buffer = new byte[1024];
    for (int i = 0; i < paths.length; i++) {
        files[i] = new File(paths[i]);
    }
    for (int i = 0; i < files.length; i++) {
        FileInputStream fis = new FileInputStream(files[i]);
        if (files.length == 1 && name != null) {
            out.putNextEntry(new ZipEntry(name));
        } else {
            out.putNextEntry(new ZipEntry(files[i].getName()));
        }
        int len;
        // 读入需要下载的文件的内容，打包到zip文件
        while ((len = fis.read(buffer)) > 0) {
            out.write(buffer, 0, len);
        }
        out.closeEntry();
        fis.close();
    }
    out.close();
}

测试

@Test
void zip() throws Exception {
    zipUtils.zip("C:\\Users\\Administrator\\Desktop\\模板.xls", "C:\\Users\\Administrator\\Desktop\\模板.zip", "1");
}

成功

解析

String input ：定义的是待压缩文件的条目。

String output：定义得到的压缩文件包.zip的名字。

String name：定义压缩后的条目的名字，如果与压缩前保持一致，定义name为null即可。

解压

实战

/**
 * 解压
 */
public void unzip(String inputFile, String destDirPath) throws Exception {
    File srcFile = new File(inputFile);//获取当前压缩文件
    // 判断源文件是否存在
    if (!srcFile.exists()) {
        throw new Exception(srcFile.getPath() + "所指文件不存在");
    }
    //开始解压
    //构建解压输入流
    ZipInputStream zIn = new ZipInputStream(new FileInputStream(srcFile), Charset.forName("GBK"));
    ZipEntry entry = null;
    File file = null;
    while ((entry = zIn.getNextEntry()) != null) {
        if (!entry.isDirectory()) {
            file = new File(destDirPath, entry.getName());
            if (!file.exists()) {
                new File(file.getParent()).mkdirs();//创建此文件的上级目录
            }
            OutputStream out = new FileOutputStream(file);
            BufferedOutputStream bos = new BufferedOutputStream(out);
            int len = -1;
            byte[] buf = new byte[1024];
            while ((len = zIn.read(buf)) != -1) {
                bos.write(buf, 0, len);
            }
            // 关流顺序，先打开的后关闭
            bos.close();
            out.close();
        }
    }
}

测试

@Test
void unzip() throws Exception {
    zipUtils.unzip("C:\\Users\\Administrator\\Desktop\\test.zip", "C:\\Users\\Administrator\\Desktop");
}

问题

java.lang.IllegalArgumentException: MALFORMED

windows环境下，默认字符集为GBK，ZipFile默认使用UTF-8字符集，当文件名存在中文时，处理时就会报错

ZipInputStream zIn = new ZipInputStream(new FileInputStream(srcFile), Charset.forName("GBK"));

2、上传/下载

文件下载

服务类

public void logDownload(String name, HttpServletResponse response) throws Exception {
    File file = new File("logs" + File.separator + name);

    if (!file.exists()) {
        throw new GlobalException(name + "文件不存在");
    }
    response.setContentType("application/force-download");
    response.addHeader("Content-Disposition", "attachment;fileName=" + name);

    byte[] buffer = new byte[1024];
    try (FileInputStream fis = new FileInputStream(file);
                  BufferedInputStream bis = new BufferedInputStream(fis)) {

        OutputStream os = response.getOutputStream();

        int i = bis.read(buffer);
        while (i != -1) {
            os.write(buffer, 0, i);
            i = bis.read(buffer);
        }
    }
}

前端控制类

/**
 * 下载日志接口
 *
 * @param name
 * @param response
 * @throws Exception
 */
@GetMapping(value = "/download/{name}")
public void logDownload(@PathVariable String name, HttpServletResponse response) throws Exception {
    logService.logDownload(name, response);
}

测试

下载成功

文件上传

服务类

public void logUpload(MultipartFile file) throws Exception {
    if (file == null || file.isEmpty()) {
        throw new Exception("未选择需上传的日志文件");
    }

    String filePath = new File("src/main/resources/static/logs").getAbsolutePath();
    File fileUpload = new File("src/main/resources/static/logs");
    if (!fileUpload.exists()) {
        fileUpload.mkdirs();
    }

    fileUpload = new File(filePath, file.getOriginalFilename());
    if (fileUpload.exists()) {
        throw new Exception("上传的日志文件已存在");
    }

    try {
        file.transferTo(fileUpload);

    } catch (IOException e) {
        throw new Exception("上传日志文件到服务器失败：" + e.toString());
    }
}

前端控制类

/**
 * 上传日志接口
 *
 * @param file
 * @return
 * @throws Exception
 */
@PostMapping(value = "/upload")
public void logUpload(@RequestParam("file") MultipartFile file) throws Exception {
    logService.logUpload(file);
}

HTML页面

错误

Circular view path [test]: would dispatch back to the current handler URL [/upload/test] again. Check your ViewResolver setup! (Hint: This may be the result of an unspecified view, due to default view name generation.)

错误原因：返回页面用@Controller，要想返回数据就用@RestController

把@controller改为@RestController就可以了；

java客户端上传文件

依赖

    <dependency>
        <groupId>commons-httpclient</groupId>
        <artifactId>commons-httpclient</artifactId>
        <version>3.1</version>
    </dependency>

新建客户端

@Test
public void logUpload() throws Exception {
    String url = "http://127.0.0.1:8080/upload";
    String pathname = new File("src/main/resources/static/logs" + File.separator + "1.log").getCanonicalPath();
    logUpload(url, pathname);
}

private static void logUpload(String url, String pathname) {
    HttpClient httpClient = new HttpClient();
    PostMethod postMethod = new PostMethod(url);

    try {
        FilePart filePart = new FilePart("file", new File(pathname));
        Part[] parts = {filePart};

        MultipartRequestEntity multipartRequestEntity = new MultipartRequestEntity(parts, new HttpMethodParams());
        postMethod.setRequestEntity(multipartRequestEntity);
        httpClient.executeMethod(postMethod);
        String result = postMethod.getResponseBodyAsString();

        System.out.println(result);
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        postMethod.releaseConnection();
    }
}

3、EasyExcel操作Excel

EasyExcel是一个基于Java的简单、省内存的读写Excel的开源项目。

官网EasyExcel

实体类

@Data
public class DemoData {

    @ExcelProperty(index = 0, value = "名字")
    private String name;

    @ExcelProperty(index = 1, value = "时间")
    private Date date;

    @ExcelProperty(index = 2, value = "数字")
    private Double num;

}

监听类

// 有个很重要的点 DemoDataListener 不能被spring管理，要每次读取excel都要new,然后里面用到spring可以构造方法传进去
@Slf4j
public class DemoDataListener implements ReadListener<DemoData> {

    //一行一行读取
    //可以把他输出进行操作
    @Override
    public void invoke(DemoData demoData, AnalysisContext analysisContext) {
        //调用方法添加数据库
        DemoData data = new DemoData();
        BeanUtils.copyProperties(demoData, data);
        log.info(data.toString());
    }

    @Override
    public void doAfterAllAnalysed(AnalysisContext analysisContext) {

    }

}

测试类

/**
 * 最简单的读
 * <p>1. 创建excel对应的实体对象 参照{@link DemoData}
 * <p>2. 由于默认一行行的读取excel，所以需要创建excel一行一行的回调监听器，参照{@link DemoDataListener}
 * <p>3. 直接读即可
 */
@Test
public void simpleRead() {
    String fileName = "src/main/resources/static/logs" + File.separator + "demo.xls";
    // 这里 需要指定读用哪个class去读，然后读取第一个sheet 文件流会自动关闭
    EasyExcel.read(fileName, DemoData.class, new DemoDataListener()).sheet().doRead();
}

/**
 * 最简单的写
 * <p>1. 创建excel对应的实体对象
 * <p>2. 直接写即可
 */
@Test
public void simpleWrite() {
        String fileName = "src/main/resources/static/logs" + File.separator + "demo.xls";
        // 这里 需要指定写用哪个class去读，然后写到第一个sheet，名字为模板 然后文件流会自动关闭
        // 如果这里想使用03 则 传入excelType参数即可
        EasyExcel.write(fileName, DemoData.class).sheet("模板").doWrite(data());
}
private List<DemoData> data() {
    List<DemoData> list = ListUtils.newArrayList();
    for (int i = 0; i < 10; i++) {
        DemoData data = new DemoData();
        data.setName("字符串" + i);
        data.setDate(new Date());
        data.setNum(0.56);
        list.add(data);
    }
    return list;
}

4、SpringBatch

先来粘一段spring官网的介绍

所有批处理都可以用最简单的形式描述为读取大量数据，执行某种类型的计算或转换，然后写出结果。Spring Batch 提供了三个关键接口来帮助执行批量读写： ItemReader、ItemProcessor和ItemWriter.

ItemReader

虽然是一个简单的概念，但它ItemReader是从许多不同类型的输入中提供数据的方法。最普遍的例子包括：

• 平面文件：
• XML：
• 数据库：

ItemReader是通用输入操作的基本接口，如下接口定义所示：

public interface ItemReader<T> {

    T read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException;

}

该read方法定义了ItemReader. 调用它会返回一个项目，或者null如果没有更多项目。

ItemWriter

ItemWriter在功能上与 ItemReader相似，但具有逆运算。资源仍然需要定位、打开和关闭，但它们的不同之处在于 ItemWriter写出而不是读入。在数据库或队列的情况下，这些操作可能是插入、更新或发送。输出的序列化格式特定于每个批处理作业。

与 一样ItemReader， ItemWriter是一个相当通用的接口，如下面的接口定义所示：

public interface ItemWriter<T> {

    void write(List<? extends T> items) throws Exception;

}

与read一样write提供了基本的合约ItemWriter。只要它处于打开状态，它就会尝试写出传入的项目列表。

ItemStream

两者都很好ItemReaders地ItemWriters服务于它们各自的目的，但是它们之间有一个共同的问题，即需要另一个接口。通常，作为批处理作业范围的一部分，需要打开、关闭读取器和写入器，并且需要一种保持状态的机制。该ItemStream接口用于此目的，如以下示例所示：

public interface ItemStream {

    void open(ExecutionContext executionContext) throws ItemStreamException;

    void update(ExecutionContext executionContext) throws ItemStreamException;

    void close() throws ItemStreamException;
}

使用,导入依赖

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jdbc</artifactId>
</dependency>
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
</dependency>

Batch有关于数据库的操作所以要导入JDBC的场景

读平面文件

@Test
public void show() throws Exception {
    FlatFileItemReader<Player> itemReader = new FlatFileItemReader<>();
    itemReader.setResource(new FileSystemResource("src/main/resources/static/logs/players.txt"));
    DefaultLineMapper<Player> lineMapper = new DefaultLineMapper<>();
    //DelimitedLineTokenizer defaults to comma as its delimiter
    lineMapper.setLineTokenizer(new DelimitedLineTokenizer());
    lineMapper.setFieldSetMapper(new PlayerFieldSetMapper());
    itemReader.setLineMapper(lineMapper);
    itemReader.open(new ExecutionContext());
    for (int i = 0; i < 6; i++) {
        Player player = itemReader.read();
        System.out.println(player);
    }
}

protected static class PlayerFieldSetMapper implements FieldSetMapper<Player> {
    public Player mapFieldSet(FieldSet fieldSet) {
        Player player = new Player();
        player.setID(fieldSet.readString(0));
        player.setLastName(fieldSet.readString(1));
        player.setFirstName(fieldSet.readString(2));
        player.setPosition(fieldSet.readString(3));
        player.setBirthYear(fieldSet.readInt(4));
        player.setDebutYear(fieldSet.readInt(5));

        return player;
    }
}

实体类

@Data
public class Player implements Serializable {

    private String ID;
    private String lastName;
    private String firstName;
    private String position;
    private int birthYear;
    private int debutYear;

    public String toString() {
        return "PLAYER:ID=" + ID + ",Last Name=" + lastName +
                ",First Name=" + firstName + ",Position=" + position +
                ",Birth Year=" + birthYear + ",DebutYear=" +
                debutYear;
    }
}

文件

AbduKa00,Abdul-Jabbar,Karim,rb,1974,1996
AbduRa00,Abdullah,Rabih,rb,1975,1999
AberWa00,Abercrombie,Walter,rb, 1959,1982
AbraDa00,Abramowicz,Danny,wr,1945,1967
AdamBo00,Adams,Bob,te,1946,1969
AdamCh00,Adams,Charlie,wr,1979,2003

写平面文件

测试类

@Autowired
FlatFileItemWriter<Player> flatFileItemWriter;

@Test
public void write() throws Exception {
    ArrayList<Player> list = new ArrayList<>();
    Player player = new Player();
    player.setID("AbduKa00");
    player.setFirstName("Abdul-Jabbar");
    player.setPosition("Karim");
    player.setLastName("rb");
    player.setBirthYear(1974);
    player.setDebutYear(1996);
    list.add(player);
    flatFileItemWriter.open(new ExecutionContext());
    flatFileItemWriter.write(list);
}

itemWriter

@Bean
public FlatFileItemWriter<Player> itemWriter() {
    return  new FlatFileItemWriterBuilder<Player>()
            .name("itemWriter")
            .resource(new FileSystemResource("src/main/resources/static/logs/players副本.txt"))
            .lineAggregator(new PassThroughLineAggregator<>())
            .build();
}

5.对象存储

YYGH系统要增加一个用户认证的功能,需要把图片上传到服务器,考虑到未来的用户比较多我们现在实验云oss服务器。这里我选用的是七牛云,因为他有免费的10G额度。首先我们要先来配置七牛云。

配置七牛云

完成实名之后创建一个存储空间,为他配置好一个域名

正好我在阿里云上有一个域名,于是准备把一个二级域名给他cdn.zhaodapiaoliang.top

这样我们的七牛云就配置好了

测试一下上传文件,没有问题[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

下面我们就需要用springboot控制上传

这是官方的SDK

先导入依赖

进行简单的上传操作

这是官方给我们的文件上传文档

Java SDK_SDK 下载_对象存储 - 七牛开发者中心 (qiniu.com)

建立service-oss服务

导入依赖

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.google.code.gson</groupId>
        <artifactId>gson</artifactId>
    </dependency>
    <dependency>
        <groupId>joda-time</groupId>
        <artifactId>joda-time</artifactId>
    </dependency>
    <dependency>
        <groupId>com.qiniu</groupId>
        <artifactId>qiniu-java-sdk</artifactId>
        <version>7.11.0</version>
    </dependency>
</dependencies>

配置文件

# ????
server.port=8205
# ???
spring.application.name=service-oss

# ?????dev?test?prod
spring.profiles.active=dev

#??json???????
spring.jackson.date-format=yyyy-MM-dd HH:mm:ss
spring.jackson.time-zone=GMT+8

# nacos????
spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848

# 七牛云配置文档
qiniuyun.access-key=???
qiniuyun.secret-key=???
qiniuyun.bucket=xiaolanlan
qiniuyun.domain=cdn.zhaodapiaoliang.top

spring.servlet.multipart.max-file-size=10MB
spring.servlet.multipart.max-request-size=10MB

配置类加载配置文件中的参数

@Component
@Data
public class qiNiuYunConfig implements InitializingBean {

    @Value("${qiniuyun.access-key}")
    private String accessKey;

    @Value("${qiniuyun.secret-key}")
    private String secretKey;

    @Value("${qiniuyun.bucket}")
    private String Bucket;

    @Value("${qiniuyun.domain}")
    private String Domain;

    public static String ACCESS_KET;
    public static String SECRET_KET;
    public static String BUCKET;
    public static String DOMAIN;
    @Override
    public void afterPropertiesSet() throws Exception {
        ACCESS_KET = accessKey;
        SECRET_KET = secretKey;
        BUCKET = Bucket;
        DOMAIN = Domain;
    }
}

下载操作service

@Service
@Slf4j
public class FileSerivceImpl implements FileService {
    @Autowired
    private qiNiuYunConfig qiniuyun;

    @Override
    public String uplodad(MultipartFile file) {

        //构造一个带指定 Region 对象的配置类
        Configuration cfg = new Configuration(Region.region1());
        UploadManager uploadManager = new UploadManager(cfg);
        //默认不指定key的情况下，以文件内容的hash值作为文件名
        String key = null;
        try {
            Auth auth = Auth.create(qiniuyun.getAccessKey(), qiniuyun.getSecretKey());
            String upToken = auth.uploadToken(qiniuyun.getBucket());

            try {
                log.info(file.getOriginalFilename());
                String uuid = UUID.randomUUID().toString().replace("-", "");
                String fileName = uuid + file.getOriginalFilename();
                String timeUrl = new DateTime().toString("yyyy/MM/dd");
                fileName = timeUrl + "/" + fileName;
                Response response = uploadManager.put(file.getInputStream(), fileName, upToken, null, null);
                //解析上传成功的结果
                DefaultPutRet putRet = new Gson().fromJson(response.bodyString(), DefaultPutRet.class);
                log.info("http://" + qiniuyun.getDomain() + "/" + putRet.key);
                return "http://" + qiniuyun.getDomain() + "/" + putRet.key;
            } catch (QiniuException ex) {
                Response r = ex.response;
                System.err.println(r.toString());
                try {
                    System.err.println(r.bodyString());
                } catch (QiniuException ex2) {
                    //ignore
                }
            }
        } catch (UnsupportedEncodingException ex) {
            //ignore
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

        return null;
    }
}

controller

@RestController
@RequestMapping("/api/oss/file")
public class FileApiController {

    @Autowired
    private FileService fileService;

    //上传文件到阿里云oss
    @PostMapping("fileUpload")
    public Result fileUpload(MultipartFile file){
        String url = fileService.uplodad(file);
        return Result.ok(url);
    }
}